Reviewerly/exHarmony huggingface dataset:
Learned Reverse ISP with Soft Supervision

RAW image serves as the foundation for camera imaging, which resides at the very beginning of the pipeline that generates sRGB images. Unfortunately, owing to special considerations, the information-rich RAW images are forfeited by default in most existing applications. To regain the RAW image, some works attempt to restore RAW images from RGB images. They focus on designing handcrafted model-based methods or complicated networks, however, ignoring the special property of RAW image, i.e., high dynamic range. To make up for this deficiency, we introduce a novel soft supervision, derived from the high dynamic range. Specifically, we propose to soften the original ground-truth as a multivariate Gaussian distribution so that networks could learn much more information. Then, we introduce a soft supervision driven network (SSDNet), based on convolution and transformer, for effectively restoring RAW images from RGB images. Quantitative and qualitative results show the promising restoration performance of RGB-to-RAW. In particular, our method achieved fifth place in the S7 track of AIM Reversed ISP Challenge. The source code will be available at https://github.com/yuezhang98/Learned-Reverse-ISP-with-Soft-Supervision .
Unsupervised Scene Sketch to Photo Synthesis

Sketches make an intuitive and powerful visual expression as they are fast executed freehand drawings. We present a method for synthesizing realistic photos from scene sketches. Without the need for sketch and photo pairs, our framework directly learns from readily available large-scale photo datasets in an unsupervised manner. To this end, we introduce a standardization module that provides pseudo sketch-photo pairs during training by converting photos and sketches to a standardized domain, i.e. the edge map. The reduced domain gap between sketch and photo also allows us to disentangle them into two components: holistic scene structures and low-level visual styles such as color and texture. Taking this advantage, we synthesize a photo-realistic image by combining the structure of a sketch and the visual style of a reference photo. Extensive experimental results on perceptual similarity metrics and human perceptual studies show the proposed method could generate realistic photos with high fidelity from scene sketches and outperform state-of-the-art photo synthesis baselines. We also demonstrate that our framework facilitates a controllable manipulation of photo synthesis by editing strokes of corresponding sketches, delivering more fine-grained details than previous approaches that rely on region-level editing.
Residual Feature Distillation Channel Spatial Attention Network for ISP on Smartphone

With the increasing popularity of mobile photography, more and more attention is being paid to image signal processing(ISP) algorithms used to improve various perceptual aspects of mobile photos. For this, a learned smartphone ISP task was proposed to develop an end-to-end deep learning-based ISP pipeline that can replace classical hand-crafted ISP and imitate target RGB images captured by digital single-lens reflex camera(DSLR). However, hardware limitations of mobile phone make it essential and challenging to achieve an acceptable trade off between computational cost and performance. In this paper, a light-weighted and powerful network named residual feature distillation channel spatial attention network (RFDCSANet) is proposed for end-to-end learned smartphone ISP task. To be specific, we employ modified residual feature distillation block(RFDB) including channel spatial attention(CSA) mechanism to progressively refine distilled features and adaptively fuse channel and spatial features. Particularly, we utilize a re-parameterizable block, namely edge-oriented convolution block(ECB) as the basic module to improve performance without introducing any additional cost in the inference stage. The proposed solution ranked $$\boldsymbol{3^{rd}}$$ in Mobile AI 2022 Learned Smartphone ISP Challenge (Track 1) with $$\boldsymbol{1^{st}}$$ place PSNR score.
LiteDepth: Digging into Fast and Accurate Depth Estimation on Mobile Devices

Monocular depth estimation is an essential task in the computer vision community. While tremendous successful methods have obtained excellent results, most of them are computationally expensive and not applicable for real-time on-device inference. In this paper, we aim to address more practical applications of monocular depth estimation, where the solution should consider not only the precision but also the inference time on mobile devices. To this end, we first develop an end-to-end learning-based model with a tiny weight size (1.4MB) and a short inference time (27FPS on Raspberry Pi 4). Then, we propose a simple yet effective data augmentation strategy, called R $$^{2}$$ crop, to boost the model performance. Moreover, we observe that the simple lightweight model trained with only one single loss term will suffer from performance bottleneck. To alleviate this issue, we adopt multiple loss terms to provide sufficient constraints during the training stage. Furthermore, with a simple dynamic re-weight strategy, we can avoid the time-consuming hyper-parameter choice of loss terms. Finally, we adopt the structure-aware distillation to further improve the model performance. Notably, our solution named LiteDepth ranks 2 $$^{nd}$$ in the MAI &AIM2022 Monocular Depth Estimation Challenge, with a si-RMSE of 0.311, an RMSE of 3.79, and the inference time is 37ms tested on the Raspberry Pi 4. Notably, we provide the fastest solution to the challenge. Codes and models will be released at https://github.com/zhyever/LiteDepth .
Towards Real-World Video Deblurring by Exploring Blur Formation Process

This paper aims at exploring how to synthesize close-to-real blurs that existing video deblurring models trained on them can generalize well to real-world blurry videos. In recent years, deep learning-based approaches have achieved promising success on video deblurring task. However, the models trained on existing synthetic datasets still suffer from generalization problems over real-world blurry scenarios with undesired artifacts. The factors accounting for the failure remain unknown. Therefore, we revisit the classical blur synthesis pipeline and figure out the possible reasons, including shooting parameters, blur formation space, and image signal processor (ISP). To analyze the effects of these potential factors, we first collect an ultra-high frame-rate (940 FPS) RAW video dataset as the data basis to synthesize various kinds of blurs. Then we propose a novel realistic blur synthesis pipeline termed as RAW-Blur by leveraging blur formation cues. Through numerous experiments, we demonstrate that synthesizing blurs in the RAW space and adopting the same ISP as the real-world testing data can effectively eliminate the negative effects of synthetic data. Furthermore, the shooting parameters of the synthesized blurry video, e.g., exposure time and frame-rate play significant roles in improving the performance of deblurring models. Impressively, the models trained on the blurry data synthesized by the proposed RAW-Blur pipeline can obtain more than 5dB PSNR gain against those trained on the existing synthetic blur datasets. We believe the novel realistic synthesis pipeline and the corresponding RAW video dataset can help the community to easily construct customized blur datasets to improve real-world video deblurring performance largely, instead of laboriously collecting real data pairs.
An 8-bit integer true periodic orbit PRNG based on delayed Arnold’s cat map

Designing a pseudo-random number generator (PRNG) exhibiting true periodic orbits with large periods is promising for embedded applications. Implementing chaotic behaviors on a finite precision digital platform using conventional unsigned integer or floating-point arithmetic yields short period limit-cycles. It leads the output to diverge from the true chaotic orbits. This paper presents an unsigned integer encoded PRNG based on the delayed quantized Arnold's Cat Map (QACM) that exhibits true periodic orbits with large periods. The proposed PRNG is obtained by delaying one of the 2-D QACM outputs with a linear feedback shift register (LFSR) thereby increasing its period. By investigating its period, we show that the periodicity increases with an increase in the delay and bit-precision. We also present the proposed PRNG unit's hardware architecture and its synthesis results targeting on a Xilinx Zynq 7000 Field Programmable Gate Array (FPGA) with 8-bit precision and delay δ=58. Further, we synthesize the design using 90 nm CMOS technology for ASIC realization. The synthesis results demonstrate the throughput as 2.208 Gbps and 5.46 Gbps at 138 MHz and 342 MHz operating frequency on FPGA and ASIC respectively. We experimentally evaluate the performance of random numbers using the NIST 800-22-1 tests along with the correlation and key sensitivity tests.
Source camera attribution via PRNU emphasis: Towards a generalized multiplicative model

The photoresponse non-uniformity (PRNU) is a camera-specific pattern, which acts as unique fingerprint of any imaging sensor and thus is widely adopted to solve multimedia forensics problems such as device identification or forgery detection. Customarily, the theoretical analysis of this fingerprint relies on a multiplicative model for the denoising residuals. This setup assumes that the nonlinear mapping from scene irradiance to preprocessed luminance, that is, the composition of the Camera Response Function (CRF) with the digital preprocessing pipeline, is a gamma correction. However, this assumption seldom holds in practice. In this paper, we improve the multiplicative model by including the influence of this nonlinear mapping, termed PRNU emphasis, on the denoising residuals. On the theoretical side, we conduct first an exploratory analysis to show that the response of typical cameras deviates from a gamma correction. We also propose a regularized least squares estimator to measure this effect. On the practical side, we argue that the PRNU emphasis is especially beneficial for a source camera attribution problem with cropped images. We back our argument with an extensive empirical evaluation using different denoisers and both compressed and uncompressed images. This new model will pave the way to future PRNU estimators and detectors.
A secure deterministic remote state preparation via a seven-qubit entangled channel of a two-qubit entangled state under the impact of quantum noise

As one of the most prominent subfields of quantum communication research, remote state preparation (RSP) plays a crucial role in quantum networks. Here we present a deterministic remote state preparation scheme to prepare an arbitrary two-qubit state via a seven-qubit entangled channel created from Borras \emph{et al.} state. Quantum noises are inherent to each and every protocol for quantum communication that is currently in use, putting the integrity of quantum communication systems and their dependability at risk. The initial state of the system was a pure quantum state, but as soon as there was any noise injected into the system, it transitioned into a mixed state. In this article, we discuss the six different types of noise models namely bit-flip noise, phase-flip noise, bit-phase-flip noise, amplitude damping, phase damping and depolarizing noise. The impact these noises had on the entangled channel may be seen by analysing the density matrices that have been altered as a result of the noise. For the purpose of analysing the impact of noise on the scheme, the fidelity between the original quantum state and the remotely prepared state has been assessed and graphically represented. In addition, a comprehensive security analysis is performed, demonstrating that the suggested protocol is safe against internal and external attacks.
Low-power continuous-wave all-optical magnetic switching in ferromagnetic nanoarrays

All-optical magnetic switching promises ultrafast magnetization control without a magnetic field. Existing schemes typically require power-hungry femtosecond-pulsed lasers and complex magnetic materials. Here, we demonstrate deterministic, all-optical magnetic switching in simple ferromagnetic nanomagnets (Ni81Fe19, Ni50Fe50) with sub-diffraction limit dimensions using a focused low-power, linearly polarized continuous-wave laser. Isolated nanomagnets are switched across a range of dimensions, laser wavelengths, and powers. All square-geometry artificial spin ice vertex configurations are written at low powers (2.74 mW). Usually, switching with linearly polarized light is symmetry forbidden; here, the laser spot has a similar size to the nanomagnets, producing an absorption distribution that depends on the nanoisland-spot displacement. We attribute the deterministic switching to the transient dynamics of this asymmetric absorption. No switching is observed in Co or Ni nanostructures, suggesting the multi-species nature of NiFe plays a role. These results usher in inexpensive, low-power, optically controlled devices with impact across data storage, neuromorphic computation, and reconfigurable magnonics.
CATS: A Serious Game in Industry Towards Stronger Cloud Security

Cloud computing has become a widely applied technology in the industry. Broad network access as a characteristic of cloud computing brings business value. It poses threats to cloud assets due to a greater attack surface than on-premises and other service models. Industry standards aim to regulate cloud security by enforcing best practices. To comply with the standards, practitioners in the industry are mandated to be trained to understand basic concepts of attack and defense mechanisms in cloud security to protect assets in the cloud. This work presents a serious game: Cloud of Assets and Threats (CATS), as an enrichment to the traditional training material to raise awareness about the cloud security challenges. In this paper, we introduce the design elements and implementation details of CATS. We organized eight game events with 94 industrial practitioners to validate our design. We applied a questionnaire and conducted semi-structured interviews with the game participants to evaluate the impact of the game and collect feedback. The evaluation indicates that CATS is a promising innovative method for promoting awareness of cloud security issues among practitioners in the industry, regardless of their technical background. Our main contributions are the design of such a game and the understanding of the impact of playing the CATS game in the industry.
Semi-Equivariant GNN Architectures for Jet Tagging

Composing Graph Neural Networks (GNNs) of operations that respect physical symmetries has been suggested to give better model performance with a smaller number of learnable parameters. However, real-world applications, such as in high energy physics have not born this out. We present the novel architecture VecNet that combines both symmetry-respecting and unconstrained operations to study and tune the degree of physics-informed GNNs. We introduce a novel metric, the \textit{ant factor}, to quantify the resource-efficiency of each configuration in the search-space. We find that a generalized architecture such as ours can deliver optimal performance in resource-constrained applications.
A light‐weight stereo matching network based on multi‐scale features fusion and robust disparity refinement

Abstract In recent years, convolutional‐neural‐network based stereo matching methods have achieved significant gains compared to conventional methods in terms of both speed and accuracy. Current state‐of‐the‐art disparity estimation algorithms require many parameters and large amounts of computational resources and are not suited for applications on edge devices. In this paper, an end‐to‐end light‐weight network (LWNet) for fast stereo matching is proposed, which consists of an efficient backbone with multi‐scale feature fusion for feature extraction, a 3D U‐Net aggregation architecture for disparity computation, and color guidance in a 2D convolutional neural network (CNN) for disparity refinement. MobileNetV2 is adopted as an efficient backbone in feature extraction. The channel attention module is applied to improve the representational capacity of features and multi‐resolution information is adaptively incorporated into the cost volume via cross‐scale connections. Further, a left‐right consistency check and color guidance refinement are introduced and a robust disparity refinement network is designed with skip connections and dilated convolutions to capture global context information and improve disparity estimation accuracy with little computational cost and memory space. Extensive experiments on Scene Flow, KITTI 2015, and KITTI 2012 demonstrate that the proposed LWNet achieves competitive accuracy and speed when compared with state‐of‐the‐art stereo matching methods.
Explainability of High Energy Physics events classification using SHAP

Abstract Complex machine learning models have been fundamental for achieving accurate results regarding events classification in High Energy Physics (HEP). However, these complex models or black-box systems lack transparency and interpretability. In this work, we use the SHapley Additive exPlanations (SHAP) method for explaining the output of two event machine learning classifiers, based on eXtreme Gradient Boost (XGBoost) and deep neural networks (DNN). We compute SHAP values to interpret the results and analyze the importance of individual features, and the experiments show that SHAP method has high potential for understanding complex machine learning model in the context of high energy physics.
An Optimization-Linked Intelligent Security Algorithm for Smart Healthcare Organizations

IoT-enabled healthcare apps are providing significant value to society by offering cost-effective patient monitoring solutions in IoT-enabled buildings. However, with a large number of users and sensitive personal information readily available in today's fast-paced, internet, and cloud-based environment, the security of these healthcare systems must be a top priority. The idea of safely storing a patient's health data in an electronic format raises issues regarding patient data privacy and security. Furthermore, with traditional classifiers, processing large amounts of data is a difficult challenge. Several computational intelligence approaches are useful for effectively categorizing massive quantities of data for this goal. For many of these reasons, a novel healthcare monitoring system that tracks disease processes and forecasts diseases based on the available data obtained from patients in distant communities is proposed in this study. The proposed framework consists of three major stages, namely data collection, secured storage, and disease detection. The data are collected using IoT sensor devices. After that, the homomorphic encryption (HE) model is used for secured data storage. Finally, the disease detection framework is designed with the help of Centered Convolutional Restricted Boltzmann Machines-based whale optimization (CCRBM-WO) algorithm. The experiment is conducted on a Python-based cloud tool. The proposed system outperforms current e-healthcare solutions, according to the findings of the experiments. The accuracy, precision, F1-measure, and recall of our suggested technique are 96.87%, 97.45%, 97.78%, and 98.57%, respectively, according to the proposed method.
SECDFAN: A Cyber Threat Intelligence System for Discussion Forums Utilization

Cyber Threat intelligence (CTI) systems offer new capabilities in the arsenal of information security experts, who can explore new sources of data that were partially exploited during the past decades. This paper deals with the exploitation of discussion forums as a source of raw data for a cyber threat intelligence process. Specifically, it analyzes the discussion forums’ characteristics and investigates their relationship with CTI. It proposes a semantic schema for the representation of data collected from discussion forums. Then, it applies a systematic methodology to design the reference architecture of the SECDFAN system, which handles the creation of CTI products following a comprehensive approach from the source selection to CTI product sharing and security experts’ collaboration. The final product of this work is the SECDFAN reference architecture. The contribution of this paper is the development of a CTI reference architecture of a system that, by design, handles all CTI-related issues for creating CTI products by analyzing the content of discussion forums.
Breast Abnormality Boundary Extraction in Mammography Image Using Variational Level Set and Self-Organizing Map (SOM)

A mammography provides a grayscale image of the breast. The main challenge of analyzing mammography images is to extract the region boundary of the breast abnormality for further analysis. In computer vision, this method is also known as image segmentation. The variational level set mathematical model has been proven to be effective for image segmentation. Several selective types of variational level set models have recently been formulated to accurately segment a specific object on images. However, these models are incapable of handling complex intensity inhomogeneity images, and the segmentation process tends to be slow. Therefore, this study formulated a new selective type of the variational level set model to segment mammography images that incorporate a machine learning algorithm known as Self-Organizing Map (SOM). In addition to that, the Gaussian function was applied in the model as a regularizer to speed up the processing time. Then, the accuracy of the segmentation’s output was evaluated using the Jaccard, Dice, Accuracy and Error metrics, while the efficiency was assessed by recording the computational time. Experimental results indicated that the new proposed model is able to segment mammography images with the highest segmentation accuracy and fastest computational speed compared to other iterative models.
Translation of Medical AR Research into Clinical Practice

Translational research is aimed at turning discoveries from basic science into results that advance patient treatment. The translation of technical solutions into clinical use is a complex, iterative process that involves different stages of design, development, and validation, such as the identification of unmet clinical needs, technical conception, development, verification and validation, regulatory matters, and ethics. For this reason, many promising technical developments at the interface of technology, informatics, and medicine remain research prototypes without finding their way into clinical practice. Augmented reality is a technology that is now making its breakthrough into patient care, even though it has been available for decades. In this work, we explain the translational process for Medical AR devices and present associated challenges and opportunities. To the best knowledge of the authors, this concept paper is the first to present a guideline for the translation of medical AR research into clinical practice.