Reviewerly/exHarmony huggingface dataset:
Progressive refinement imaging with depth-assisted disparity correction

In recent years, the increasing on-board compute power of mobile camera devices gave rise to a class of digitization algorithms that dynamically fuse a stream of camera observations into a progressively updated scene representation. Previous algorithms either obtain general 3D surface representations, often exploiting range maps from a depth camera, such as, Kinect Fusion, etc.; or they reconstruct planar (or distant spherical, respectively) 2D images with respect to a single (perspective or orthographic) reference view, such as, panoramic stitching or aerial mapping. Our work sets out to combine aspects of both, reconstructing a 2.5-D representation (color and depth) as seen from a fixed viewpoint, at spatially variable resolution. Inspired by previous work on “progressive refinement imaging”, we propose a hierarchical representation that enables progressive refinement of both colors and depths by ingesting RGB-D images from a handheld depth camera that is carried through the scene. We evaluate our system by comparing it against state-of-the-art methods in 2D progressive refinement and 3D scene reconstruction, using high-detail indoor and outdoor data sets comprising medium to large disparities. As we will show, the restriction to 2.5-D from a fixed viewpoint affords added robustness (particularly against self-localization drift, as well as backprojection errors near silhouettes), increased geometric and photometric fidelity, as well as greatly improved storage efficiency, compared to more general 3D reconstructions. We envision that our representation will enable scene exploration with realistic parallax from within a constrained range of vantage points, including stereo pair generation, visual surface inspection, or scene presentation within a fixed VR viewing volume.
Visual Secret Sharing of Gray and Color Images using Fuzzy Random Grids

Visual secret sharing is a method used to secure images by converting a secret image into multiple shares so that the original image cannot be recognized from individual shares, and no calculations are required during the image decryption, but the original image can be visually seen by stacking the shares. However, one of the significant challenges in this method is the increased size of the shares and the recovered image, which is called pixel expansion. Researchers have proposed several methods to solve this problem, but many of them are limited to binary images, and they cannot handle gray and color images without converting them to binary. This study proposes a novel fuzzy random grid-based method to directly encrypt gray and color images without any pixel expansions. In this method, fuzzy random grids with decimal values between 0.0 to 1.0 are generated during the encryption stage, and fuzzy operators are used for the decryption stage. The evaluation results demonstrate the ability of the proposed solution in encrypting gray and color images without converting them to binary and without any pixel expansions. The individual shares of the proposed method do not reveal any information from the original image, making it a secure method. The quality of the decrypted images has been evaluated using both subjective and objective evaluation methods such as PSNR and SSIM metrics, which show state-of-the-art results.
MEGACare: Knowledge-guided multi-view hypergraph predictive framework for healthcare

Predicting a patient's future health condition by analyzing their Electronic Health Records (EHRs) is a trending subject in the intelligent medical field, which can help clinicians prescribe safely and effectively, and also make more accurate diagnoses. Benefiting from powerful feature extraction capabilities, graph representation learning can capture complex relationships and achieve promising performance in many clinical prediction tasks. However, existing works either exclusively consider single domain knowledge with an independent task or do not fully capitalize on domain knowledge that can provide more predictive signals in the code encoding stage. Moreover, the heterogeneous and high-dimensional nature of EHR data leads to a deficiency of hardly encoding implicit high-order correlations. To address these limitations, we proposed a knowledge-guided Multi-viEw hyperGrAph predictive framework (MEGACare) for diagnosis prediction and medication recommendation. Our MEGACare leveraged multi-faceted medical knowledge, including ontology structure, code description, and molecular information to enhance medical code presentations. Furthermore, we constructed an EHR hypergraph and a multi-view learning framework to capture the high-order correlation between patient visits and medical codes. Specifically, we propose three perspectives around the pairwise relationship between patient visits and medical codes to comprehensively learn patient representation and enhance the robustness of our framework. We evaluated our MEGACare framework against a set of state-of-the-art methods for two clinical outcome prediction tasks in the public MIMIC-III dataset, and the results showed that our proposed framework was superior to the baseline methods.1
Employees' in-role and extra-role information security behaviors from the P-E fit perspective

Organizations are increasingly seeking ways to encourage employee in-role and extra-role information security (InfoSec) behaviors for enhancing organizational InfoSec. One important consideration is the InfoSec congruence between the person and the organizational environment. Accordingly, drawing on person–environment (P–E) fit theory, we conducted an empirical investigation that examined (1) the effect of key components in organizational InfoSec management practice on InfoSec value P–E fit, and (2) the role of InfoSec value P–E fit on employees’ InfoSec in-role and extra-role behaviors in the organization. The results suggest that three practices—InfoSec policies; security education, training, and awareness programs; and computer monitoring—cultivate InfoSec value P–E fit, leading to an increase in employees’ InfoSec in-role and extra-role behaviors.
Ensemble anomaly score for video anomaly detection using denoise diffusion model and motion filters

Video anomaly detection is a crucial task that aims to differentiate between normal and abnormal events. The current mainstream approach involves constructing an anomaly score based on the reconstruction error from a prediction model trained on normal frame sequences. However, this approach is limited by its deterministic nature, which may cause the anomaly score to be sensitive to underlying noise in the video. To address this limitation, this paper proposes an ensemble anomaly score constructed using a series of stochastic reconstructions of the original prediction. Specifically, we introduce the denoise diffusion model as a perturbation-denoise tool. First, the original prediction undergoes a perturbation process through a diffusion process. Then, a denoise diffusion model trained on normal predictions is used to directly reconstruct a series of noise-free predictions from the perturbed versions with different noise levels. Finally, an ensemble of all the reconstruction errors is used to provide a more generic and regularized anomaly score. Furthermore, we introduce motion filters into the detection pipeline to improve the modeling accuracy of the image distribution. The proposed method is evaluated on public datasets, and experimental results demonstrate its effectiveness, particularly in detecting performance under out-of-distribution (OOD) conditions.
Adaptive ResNet Architecture for Distributed Inference in Resource-Constrained IoT Systems

As deep neural networks continue to expand and become more complex, most edge devices are unable to handle their extensive processing requirements. Therefore, the concept of distributed inference is essential to distribute the neural network among a cluster of nodes. However, distribution may lead to additional energy consumption and dependency among devices that suffer from unstable transmission rates. Unstable transmission rates harm real-time performance of IoT devices causing low latency, high energy usage, and potential failures. Hence, for dynamic systems, it is necessary to have a resilient DNN with an adaptive architecture that can downsize as per the available resources. This paper presents an empirical study that identifies the connections in ResNet that can be dropped without significantly impacting the model’s performance to enable distribution in case of resource shortage. Based on the results, a multi-objective optimization problem is formulated to minimize latency and maximize accuracy as per available resources. Our experiments demonstrate that an adaptive ResNet architecture can reduce shared data, energy consumption, and latency throughout the distribution while maintaining high accuracy.
A Novel Pipeline for Improving Optical Character Recognition through Post-processing Using Natural Language Processing

Optical Character Recognition (OCR) technology finds applications in digitizing books and unstructured documents, along with applications in other domains such as mobility statistics, law enforcement, traffic, security systems, etc. The state-of-the-art methods work well with the OCR with printed text on license plates, shop names, etc. However, applications such as printed textbooks and handwritten texts have limited accuracy with existing techniques. The reason may be attributed to similar-looking characters and variations in handwritten characters. Since these issues are challenging to address with OCR technologies exclusively, we propose a post-processing approach using Natural Language Processing (NLP) tools. This work presents an end-to-end pipeline that first performs OCR on the handwritten or printed text and then improves its accuracy using NLP.
Effective deep learning-based channel state estimation and signal detection for OFDM wireless systems

Deep learning (DL) algorithms can enhance wireless communication system efficiency and address numerous physical layer challenges. Channel state estimation (CSE) and signal detection (SD) are essential parts of improving the performance of an OFDM wireless system. In this context, we introduce a DL model as an effective alternative for implicit CSE and SD over Rayleigh fading channels in the OFDM wireless system. The DL model is based on the gated recurrent unit (GRU) neural network. The proposed DL GRU model is trained offline using the received OFDM signals related to the transmitted data symbols and added pilot symbols as inputs. Then, it is implemented online to accurately and directly detect the transmitted data. The experimental results using the metric parameter of symbol error rate show that, the proposed DL GRU-based CSE/SD provides superior performance compared with the traditional least square and minimum mean square error estimation methods. Also, the trained DL GRU model exceeds the existing DL channel estimators. Moreover, it provides the highest CSE/SD quality with fewer pilots, short/null cyclic prefixes, and without prior knowledge of the channel statistics. As a result, the proposed DL GRU model is a promising solution for CSE/SD in OFDM wireless communication systems.
A Coupled Oscillator Network to Solve Combinatorial Optimization Problems with Over 95% Accuracy

Solving optimization problems is becoming increasingly important for many applications in today's world. Various software solvers and digital hardware accelerators are designed to speed up the computation while decreasing energy consumption. However, NP-complete problems like the maximum-cut are still inefficient to solve using traditional computing methods. The usage of specialized analog coupled oscillator networks is an upcoming fast and energy-efficient option. In order to compete with digital counterparts, a similar solution accuracy at significantly increased speed is required. However, reaching a very good solution with such analog-oriented principles is still a major challenge. The system proposed here focuses on a best possible accuracy and speed, while area and power consumption are of minor priority. Experimental benchmarking shows that the computed solutions reach 95% accuracy and more of commercial software, while the actual computation takes just 714 nanoseconds.
An FPGA-Friendly Algorithm for QR Code Detection

QR code is widely used in different applications, and its detection is currently being done by software. However, hardware detection using FPGAs offers real-time processing ability, which makes it attractive for time-critical applications, such as high-precision robotics and augmented reality. In light of this, an FPGA algorithm for QR code detection is proposed in this paper. It operates with a maximum latency of 12.2 ms to detect a QR code when the input image resolution is <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$640\times 480$</tex> , which offers a 85.3% performance boost over the best state-of-the-art software detector according to benchmarks. To the best of the authors' knowledge, this is the first work that explores the use of FPGA in QR code detection.
Multi-Task Learning for Screen Content Image Coding

With the rise of remote work and collaboration, compression of screen content images (SCI) is becoming increasingly important. While there are efficient codecs for natural images, as well as codecs for purely-synthetic images, those SCIs that contain both synthetic and natural content pose a particular challenge. In this paper, we propose a learning-based image coding model developed for such SCIs. By training an encoder to provide a latent representation suitable for two tasks – input reconstruction and synthetic/natural region segmentation – we create an effective SCI image codec whose strong performance is verified through experiments. Once trained, the second task (segmentation) need not be used; the codec still benefits from the segmentation-friendly latent representation.
Non-Uniform Interpolation in Integrated Gradients for Low-Latency Explainable-AI

There has been a surge in Explainable-AI (XAI) methods that provide insights into the workings of Deep Neural Network (DNN) models. Integrated Gradients (IG) is a popular XAI algorithm that attributes relevance scores to input features commensurate with their contribution to the model's output. However, it requires multiple forward & backward passes through the model. Thus, compared to a single forward-pass inference, there is a significant computational overhead to generate the explanation which hinders real-time XAI. This work addresses the aforementioned issue by accelerating IG with a hardware-aware algorithm optimization. We propose a novel non-uniform interpolation scheme to compute the IG attribution scores which replaces the baseline uniform interpolation. Our algorithm significantly reduces the total interpolation steps required without adversely impacting convergence. Experiments on the ImageNet dataset using a pre-trained InceptionV3 model demonstrate 2.6-3.6×performance speedup on GPU systems for iso-convergence. This includes the minimal 0.2-3.2% latency overhead introduced by the pre-processing stage of computing the non-uniform interpolation step-sizes.
AUC: Accountable Universal Composability

Accountability is a well-established and widely used security concept that allows for obtaining undeniable cryptographic proof of misbehavior, thereby incentivizing honest behavior. There already exist several general purpose account-ability frameworks for formal game-based security analyses. Unfortunately, such game-based frameworks do not support modular security analyses, which is an important tool to handle the complexity of modern protocols.Universal composability (UC) models provide native support for modular analyses, including re-use and composition of security results. So far, accountability has mainly been modeled and analyzed in UC models for the special case of MPC protocols, with a general purpose accountability framework for UC still missing. That is, a framework that among others supports arbitrary protocols, a wide range of accountability properties, handling and mixing of accountable and non-accountable security properties, and modular analysis of accountable protocols.To close this gap, we propose AUC, the first general purpose accountability framework for UC models, which supports all of the above, based on several new concepts. We exemplify AUC in three case studies not covered by existing works. In particular, AUC unifies existing UC accountability approaches within a single framework.
Telepath: A Minecraft-based Covert Communication System

Covert, censorship-resistant communication in the presence of nation-state adversaries requires unobservable channels whose operation is difficult to detect via network-traffic analysis. Traffic substitution, i.e., replacing data transmitted by a "cover" application with covert content, takes advantage of already-existing encrypted channels to produce traffic that is statistically indistinguishable from the traffic of the cover application and thus difficult to censor.Online games are a promising platform for building circumvention channels due to their popularity in many censored regions. We show, however, that previously proposed traffic substitution methods cannot be directly applied to games. Their traces, even if statistically similar to game traces, may violate game-specific invariants and are thus easy to detect because they could not have been generated by an actual gameplay.We explain how to identify non-disruptive content whose substitution does not result in client-server inconsistencies and use these ideas to design and implement Telepath, a covert communication system that uses Minecraft as the platform. Telepath takes advantage of (1) Minecraft’s encrypted client-server channel, (2) decentralized architecture that enables individual users to run their own servers, and (3) popularity of "mods" that add functionality to Minecraft clients and servers. Telepath runs a Minecraft game but substitutes non-disruptive in-game messages with covert content, without changing the game’s interaction with the network manager.We measure performance of Telepath for Web browsing and audio streaming, and show that network traffic generated by Telepath resists statistical traffic analysis that aims to distinguish it from popular Minecraft bots.
SNAP: Efficient Extraction of Private Properties with Poisoning

Property inference attacks allow an adversary to extract global properties of the training dataset from a machine learning model. Such attacks have privacy implications for data owners sharing their datasets to train machine learning models. Several existing approaches for property inference attacks against deep neural networks have been proposed [1] –[3], but they all rely on the attacker training a large number of shadow models, which induces a large computational overhead.In this paper, we consider the setting of property inference attacks in which the attacker can poison a subset of the training dataset and query the trained target model. Motivated by our theoretical analysis of model confidences under poisoning, we design an efficient property inference attack, SNAP, which obtains higher attack success and requires lower amounts of poisoning than the state-of-the-art poisoning-based property inference attack by Mahloujifar et al. [3]. For example, on the Census dataset, SNAP achieves 34% higher success rate than [3] while being 56.5× faster. We also extend our attack to infer whether a certain property was present at all during training and estimate the exact proportion of a property of interest efficiently. We evaluate our attack on several properties of varying proportions from four datasets and demonstrate SNAP’s generality and effectiveness.
TrojanModel: A Practical Trojan Attack against Automatic Speech Recognition Systems

While deep learning techniques have achieved great success in modern digital products, researchers have shown that deep learning models are susceptible to Trojan attacks. In a Trojan attack, an adversary stealthily modifies a deep learning model such that the model will output a predefined label whenever a trigger is present in the input. In this paper, we present TrojanModel, a practical Trojan attack against Automatic Speech Recognition (ASR) systems. ASR systems aim to transcribe voice input into text, which is easier for subsequent downstream applications to process. We consider a practical attack scenario in which an adversary inserts a Trojan into the acoustic model of a target ASR system. Unlike existing work that uses noise-like triggers that will easily arouse user suspicion, the work in this paper focuses on the use of unsuspicious sounds as a trigger, e.g., a piece of music playing in the background. In addition, TrojanModel does not require the retraining of a target model. Experimental results show that TrojanModel can achieve high attack success rates with negligible effect on the target model’s performance. We also demonstrate that the attack is effective in an over-the-air attack scenario, where audio is played over a physical speaker and received by a microphone.
From 5G Sniffing to Harvesting Leakages of Privacy-Preserving Messengers

We present the first open-source tool capable of efficiently sniffing 5G control channels, 5GSniffer and demonstrate its potential to conduct attacks on users privacy. 5GSniffer builds on our analysis of the 5G RAN control channel exposing side-channel leakage. We note that decoding the 5G control channels is significantly more challenging than in LTE, since part of the information necessary for decoding is provided to the UEs over encrypted channels. We devise a set of techniques to achieve real-time control channels sniffing (over three orders of magnitude faster than brute-forcing). This enables, among other things, to retrieve the Radio Network Temporary Identifiers (RNTIs) of all users in a cell, and perform traffic analysis. To illustrate the potential of our sniffer, we analyse two privacy-focused messengers, Signal and Telegram. We identify privacy leaks that can be exploited to generate stealthy traffic to a target user. When combined with 5GSniffer, it enables stealthy exposure of the presence of a target user in a given location (solely based on their phone number), by linking the phone number to the RNTI. It also enables traffic analysis of the target user. We evaluate the attacks and our sniffer, demonstrating nearly 100% accuracy within 30 seconds of attack initiation.
μSwitch: Fast Kernel Context Isolation with Implicit Context Switches

Isolating application components is crucial to limit the exposure of sensitive data and code to vulnerabilities in the untrusted components. Process-based isolation is the de facto isolation used in practice, e.g., web browsers. However, it incurs significant performance overhead and is typically infeasible when frequent switches between isolation domains are expected. To address this problem, many intra-process memory isolation techniques have been proposed using novel kernel abstractions, recent CPU extensions (e.g., Intel <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">®</sup> MPK), and software-based fault isolation (e.g., WebAssembly). However, these techniques insufficiently isolate kernel resources, such as file descriptors, or do so by incurring high overheads when resources are accessed. Other work virtualizes the kernel context inside a privileged user space domain, but this is ad-hoc, error-prone, and provides only limited kernel functionalities.We propose μSwitch, an efficient kernel context isolation mechanism with memory protection that addresses these limitations. We use a protected structure, shared by the kernel and the user space, for context switching and propose implicit context switching to improve its performance by deferring the kernel resource switch to the next system call. We apply μSWITCH to isolate libraries in the Firefox web browser and an HTTP server, and reduce the overhead of isolation by 32.7% to 98.4% compared with other isolation techniques.