Reviewerly/exHarmony huggingface dataset:
BackLip: Passphrase-Independent Lip-reading User Authentication with Backscatter Signals

User authentication is essential for threat defense and data protection. Existed authentication systems have some known limitations, such as spoofing attacks, privacy leakage, and user-unfriendliness. In this paper, we propose BackLip, a novel anti-spoofing authentication system based on lip reading. We employ Wi-Fi backscatter-based technology to recognize users lip reading due to its various advantages, e.g. privacy protection, low power consumption, and low cost. Our system is touch-free and passphrase-independent, making it user-friendly, especially for the elderly and disabled. We first filter out irrelevant interference and enhance the signal-to-noise ratio of lip-reading signals by modulating the backscatter tags and constructing a series of suitable filters. Then, we study the energy distribution and steady-state characteristics of the backscattered signal caused by lip-reading movement at different frequencies to extract passphrase-independen lip-reading fingerprints. We build a theoretical model to analyze and prove the feasibility of our method and design an adaptive correction method to resist the interference caused by distance and angle changes. Additionally, we propose an adaptive segmentation algorithm to label lip-reading motions automatically. Extensive experiments demonstrate that BackLip has an average accuracy of 92.1% and is improved to 96.5% when users use the same passphrases.
WAEVSR: Enabling Collaborative Live Video Super-Resolution in Wide-Area MEC Environment

Live video streaming is increasingly popular for its rich content and real-time interactions, but its demand for bandwidth has put a heavy burden on backbone networks. To save bandwidth, recent studies have proposed neural-enhanced live video streaming that deploys deep neural networks (DNNs) for video super-resolution (VSR) on end devices or nearby edge devices to enhance video quality by taking low-resolution frames as input and producing high-resolution output frames. In this solution, the high computational demands of high-quality VSR DNNs make them difficult to support on single end or edge device, necessitating the use of distributed resources in edge facilities. However, the distributed deployment of high-quality VSR DNNs for low-latency inference remains challenging due to the inherent data dependencies of VSR DNNs and the heterogeneity and dynamics of edge facilities. In this paper, we present WAEVSR, a novel collaborative neural-enhanced live video super-resolution system that enables effective leverage of distributed resources to maximize the latency-bounded quality in wide-area MEC environments. WAEVSR consists of two key components: 1) It deploys a parallel-friendly video super-resolution DNN among edge devices, 2) with an inference controller based on the variable-size sliding window to balance the latency and quality of distributed inference in the heterogeneous and dynamics MEC environment. Prototype-based evaluation shows that WAEVSR can achieve 2.5 × lower end-to-end latency than traditional super-resolution serving with a 0.01 drop in SSIM score. The case study also demonstrates its higher stability on latency than vanilla distributed MEC deployment.
AdaCoOpt: Leverage the Interplay of Batch Size and Aggregation Frequency for Federated Learning

Federated Learning (FL) is a distributed learning paradigm that can coordinate heterogeneous edge devices to perform model training without sharing private raw data. Many prior works have analyzed the FL convergence with respect to important hyperparameters, including batch size and aggregation frequency. However, adjusting the batch size and the number of local updates can affect the model performance, training time, and the cost of consuming computation and communication resources, in different and perhaps complex forms. Their joint effects have been overlooked and should be exploited to achieve accurate models with controllable operational expenditure. This paper proposes novel analytical models and optimization algorithms that leverage the interplay of batch size and aggregation frequency to navigate the trade-offs among convergence, cost, and completion time for FL. We first obtain a new convergence bound of the training error under heterogeneous training datasets across devices. Based on this bound, we derive closed-form solutions of a co-optimized batch size and aggregation frequency, a single configuration for all the devices. We then design an efficient exact algorithm for assigning different batch configurations across devices that can further improve the model accuracy to address the heterogeneity of both data and system characteristics. Further, we propose an adaptive control algorithm to dynamically adjust the solutions with estimated network states. Extensive experiments demonstrate the superiority of our offline optimal solutions and online adaptive algorithm.
A Fast Adversarial Sample Detection Approach for Industrial Internet-of-Things Applications

Adversarial attacks reveal the inherent vulnerability of deep neural networks, which face serious security issues for their security. Among them, the attack against the Deep Neural Network (DNN) application used in the Industrial Internet of Things (IIoT) is a key area in adversarial attacks. Adversarial examples generated by attackers by adding human-undetectable interference to legitimate examples may cause models to make wrong decision results, resulting in serious accidents. Many detection technologies have been proposed to mitigate the harm of adversarial examples to neural networks, among which the methods based on the difference of feature attribution between normal examples and adversarial examples show state-of-the-art detection performance, but they suffer from detection efficiency. In this work, we focus on improving the detection efficiency of the feature-attribution-based detection methods. We observe that there is still a significant difference in the feature attribution distribution of a normal image and an adversarial image even only some pixels in the image are processed, which can be verified by utilizing the Kolmogorov-Smirnov test. Based on this observation, we first adopt a variety of strategies to sample partial pixels in an image and then utilize the selected pixels to train a feature-attribution-based detector for detecting adversarial examples. Extensive experiments conducted on four datasets (MNIST, CIFAR-10, SVHN, CIFAR-100) against various attacks proved that the detection efficiency of the accelerated detection method is improved (for example, the average execution time was increased by 8.7 times on CIFAR-10) while the detection performance maintains state-of-the-art.
Federated Learning with Adaptive Weighted Model Aggregation

When using federated learning (FL) to predict user preferences, not all data samples are equally useful. For example, in video recommendation, recent watch history is more useful than older watch history. Thus, a new FL approach with adaptive weighted model aggregation (FLAMA) is proposed, where model aggregation weights are recalculated each FL training round based on the number of useful data samples used by each client and global model performance. Additionally, training datasets are compiled with a higher preference for useful data samples via prioritized data sample selection (PDSS). It is demonstrated that the proposed FLAMA achieves a higher accuracy in less FL training rounds compared to other schemes.
Multi-Layer Collaborative Bandit for Multivariate Time Series Anomaly Detection

Multivariate Time Series Anomaly Detection (MTSAD) detects abnormal indicators from Multivariate Time Series (MTS), and provides the rank of the multiple abnormal indicators to meet the expert's detection interest in current environment, which underpin the security and stability of intelligent cyber-physical systems. However, popular integration-based methods, which are pre-defined, fall short in locating the exact abnormal indicator, nor can they perceive the environmental dynamic and evolve accordingly. Let alone meeting the expert's interest. As a result, the expert's workload is exaggerated. These issues motivate us to propose a novel multi-layer collaborative bandit framework MULA for MTSAD. MULA decomposes MTS and pairs individual time series with a bandit arm, which locates the abnormal indicator directly. Then, MULA sorts the indicators by abnormal scores computed based on the expert's feedback, which facilitates the experts. Besides, to address the adaptability issue, we devise a dual signal to comprehensively monitor environmental changes, and design a multi-layer collaborative mechanism for MULA to adapt to the dynamic environment. Theoretical analysis and experiments on public datasets demonstrate the superiority of MULA compared to the state-of-art.
FTM-RCA: A Fast Two-Stage Multi-dimensional Root-Cause Analysis of Network Anomalies

Multi-dimensional Root Cause Analysis (RCA) is often applied to identify abnormal traffic patterns, i.e., localizing the abnormal combination of traffic header fields. Several techniques have been proposed recently, but they were mainly designed for smaller-scaled datasets and were not feasible in the real network due to the high computational overhead. To overcome the aforementioned limitations, we propose FTM-RCA, which accelerates RCA by breaking the analysis procedure into two stages: coarse-grained rules filtering and fine-grained localization. In the first stage, an optimized frequent itemset mining (FIM) technique called CUSC is proposed, which can detect high-volume combinations faster based on the mutual exclusion of dimension values. Experiments on CUSC show that it can speed up by 44.87% and reduce memory consumption by 21.89% compared to the best previous FIM algorithms. In the second stage, a dimension-based search method is proposed to identify the root cause combinations, which consists of two key components: 1) drill-down strategy, which utilizes Contributive Power to measure the correlation between the combination and anomaly. 2) pruning strategy, which adopts the Shannon entropy to avoid generating trivial results. As a result, the overall diagnostic time of FTM-RCA is at least 25 times faster than the previous best research while improving accuracy by an average of 21.6%. Also, our practical application in real network also illustrates the applicability of FTM-RCA.
Distributed Model Interpretation for Vertical Federated Learning with Feature Discrepancy

Vertical federated learning (VFL) allows multiple clients with misaligned feature spaces to collaboratively accomplish the global model training. Applying VFL to high stakes decision scenarios greatly requires model interpretation for decision reliability and diagnosis. However, the feature discrepancy in VFL raises new issues for model interpretation in distributed setting: one is from the local-global perspective, where the local importance of features is not equal to the global importance; and the other is from the local-local perspective, where information asymmetry among clients causes difficulty in identifying overlapped features. In this work, we propose a new distributed Model Interpretation method for Vertical Federated Learning with feature discrepancy, namely MI-VFL. In particular, to deal with the local-global discrepancy, MI-VFL leverages the law of total probability to adjust the local importance of features and ensures the completeness of the selected features using adversarial game. To handle the local-local discrepancy, MI-VFL builds a federated adversarial learning model to efficiently identify the overlapped features once, rather than performing client-to-client intersections multiple times. We extensively evaluate MI-VFL on six synthetic datasets and five real-world datasets. The evaluation results reveal that MI-VFL can accurately identify the important features, suppress the overlapped features, and thus improve the model performance.
Free Speech or Free to Hate?: Anti-LGBTQ+ Discourses in LGBTQ+-Affirming Spaces on Gab Social

This article presents a critical discourse analysis of groups on the free speech social media platform Gab that were intended to be LGBTQ+-friendly but became spaces of queerphobia. Results indicate that Gab users deployed discourses of difference to situate the platform as heteronormative and to denigrate the LGBTQ+ community. In particular, discourses utilized in the name of free speech were used to establish LGBTQ+ individuals as abominations, undergird hegemonic masculinity, and marginalize queer folk by reducing them to sex acts and sex organs. This study provides a better understanding of the (in)efficacy of "free speech" as a content moderation policy and unpacks how anti-LGBTQ+ hate speech spreads in digital spaces.
Changes in Navigation Controls and Field-of-View Modes Affect Cybersickness Severity and Spatiotemporal Gait Patterns After Exposure to Virtual Environments

To examine the effects of navigation controls and field-of-view modes on cybersickness severity and gait dynamics after cessation of exposure to a virtual environment (VE).The applications of virtual reality are increasing in various fields; however, whether changes in interaction techniques and visual contents could mitigate the potential gait disturbance following VE exposure remains unclear.Thirty healthy adults wore a head-mounted display to complete six sessions of 12-min run-and-gun tasks using different navigation controls (gamepad, head, natural) and field-of-view modes (full, restricted). Forward and backward walking tasks were performed before and after VE exposure. The degrees of cybersickness and presence were evaluated using questionnaires, along with the in-session task performance. Spatiotemporal gait measures and their variabilities were calculated for each walking task.The participants experienced less cybersickness with the head and natural controls than with the gamepad. Natural control, based on matching body movements, was associated with the highest degree of presence and best performance. VE navigation using the gamepad showed reduced cadences and increased stride times during postexposure forward-walking tasks. When the VE was presented via the restricted field-of-view mode, increased gait variabilities were observed from backward-walking tasks after VE exposure.Body movement-based navigation controls may alleviate cybersickness. We observed gait adaptation during both ambulation tasks, which was influenced by the navigation control method and field-of-view mode.This study provides the first evidence for gait adaptation during balance-demanding tasks after VE exposure, which is valuable for designing guidelines for virtual reality interactions.
An optical scheme of on-chip matrixing by phase-change based tunable weighting of photonic tensor unit

Abstract In this paper, we propose an optical scheme of on-chip matrixing for matrix-vector multiplications (MVMs) by configuring each matrix element into a photonic tensor processing unit (TPU) with wavelength division multiplexing and the actively tunable weighting for scalar multiplication. The low loss chalcogenide phase change material of Ge 2 Sb 2 Se 4 Te (GSST) is employed and modeled with intermediate states for multilevel tunable weighting of each TPU. The dynamic electro-thermal process of GSST phase transition using ITO for electrical heating is simulated and well confirms the switchable weighting of TPUs. Simulation results reveal that a 7 V voltage pulse of 500 ns duration followed by a 9 V voltage pulse of 1500 ns duration and another pulse of 14 V voltage and 500 ns duration can set data value of ‘0’ and ‘1’ for each TPU. Taking a set of incident light with varied wavelengths and powers as input vectors, the quantized MVM outputs of photocurrents with varied amplitude are obtained after photoelectric conversion. Finally, the photonic integrated circuit level simulations by Lumerical INTERCONNECT perfectly confirm our scheme of on-chip matrixing for optical MVMs and computing.
Concise and interpretable multi-label rule sets

Abstract Multi-label classification is becoming increasingly ubiquitous, but not much attention has been paid to interpretability. In this paper, we develop a multi-label classifier that can be represented as a concise set of simple “if-then” rules, and thus, it offers better interpretability compared to black-box models. Notably, our method is able to find a small set of relevant patterns that lead to accurate multi-label classification, while existing rule-based classifiers are myopic and wasteful in searching rules, requiring a large number of rules to achieve high accuracy. In particular, we formulate the problem of choosing multi-label rules to maximize a target function, which considers not only discrimination ability with respect to labels, but also diversity. Accounting for diversity helps to avoid redundancy, and thus, to control the number of rules in the solution set. To tackle the said maximization problem, we propose a 2-approximation algorithm, which circumvents the exponential-size search space of rules using a novel technique to sample highly discriminative and diverse rules. In addition to our theoretical analysis, we provide a thorough experimental evaluation and a case study, which indicate that our approach offers a trade-off between predictive performance and interpretability that is unmatched in previous work.
A Tibetan Sentence Boundary Disambiguation Model Considering the Components on Information on Both Sides of Shad

Sentence Boundary Disambiguation (SBD) is a preprocessing step for natural language processing. Segmenting text into sentences is essential for Deep Learning (DL) and pretraining language models. Tibetan punctuation marks may involve ambiguity about the sentences' beginnings and endings. Hence, the ambiguous punctuation marks must be distinguished, and the sentence structure must be correctly encoded in language models. This study proposed a component-level Tibetan SBD approach based on the DL model. The models can reduce the error amplification caused by word segmentation and part-of-speech tagging. Although most SBD methods have only considered text on the left side of punctuation marks, this study considers the text on both sides. In this study, 465 669 Tibetan sentences are adopted, and a Bidirectional Long Short-Term Memory (Bi-LSTM) model is used to perform SBD. The experimental results show that the F1-score of the Bi-LSTM model reached 96%, the most efficient among the six models. Experiments are performed on low-resource languages such as Turkish and Romanian, and high-resource languages such as English and German, to verify the models' generalization.
Sentiment Analysis Application in E-Commerce: Current Models and Future Directions

Sentiment analysis (SA), which is also known as opinion mining, is an increasingly popular practical application of Natural Language Processing (NLP). SA is especially useful in e-commerce fields, where comments and reviews often contain a wealth of valuable business information that has great research value. This study aims to investigate three related aspects of SA in e-commerce: the methods used to address the SA problem in this domain, the most commonly used e-commerce platforms where researchers get data from, and the future direction of research in this area. To achieve this goal, we reviewed 15 papers that covered many machine learning models and deep learning models. In the results and discussion section, we suggest several future directions that can improve the current SA models in this review. By addressing the limitations of existing SA models and exploring new approaches, we believe that future research in this area will lead to more accurate and effective sentiment analysis tools that can benefit both businesses and consumers.
Dissipation and irreversibility in computing

Abstract There has been much discussion for decades over the proper concepts of minimum dissipation per logic gate and what is required for computation, as well as early arguments over logically reversible machines. Here these arguments, and the thermodynamics related to them, are discussed in terms of what is required for a computing machine. This gives a set of requirements for a valid computer that arise already from Turing and a set of limitations on energy dissipation and entropy. Here, the requirements set by Turing on what can and cannot be a computing machine are introduced, and how these limits affect not only the machine, but individual bits and a general need for irreversibility, are discussed. Then, it is shown that there is a minimum dissipation for a bit operation, and this is imposed both by Turing’s requirements as well as by noise in the system. Finally, it is shown that information entropy differs from physical entropy and care must be taken in trying to connect the two quantities. How these requirements, and new ones, affect quantum computation is also discussed.
Towards Querying Multimodal Annotations Using Graphs

Photographs and 3D reconstructions of buildings as well as textual information and documents play an important role in art history and architectural studies when it comes to investigating architecture, the construction history of buildings, and the impact these constructions had on a city. Advanced tools have the potential to enhance and support research workflows and source criticism by linking corresponding materials and annotations, such that relevant data can be quickly queried and identified. Images are a primary source in the 3D reconstruction process, with the possibility to create spatializations of additional photographs of buildings which were not part of the initial SfM process, enabling the linking of annotations between these photographs and the respective 3D model. In contrast, identifying and locating respective annotations in text sources requires a different approach due to their more abstract nature. This paper presents concepts for automatic linking of texts and their respective annotations to corresponding images, as well as to 3D models and their annotations. Controlled vocabularies for architectural elements and a graph representation are utilized to reduce ambiguity when querying related instances.
DeepSHAP Summary for Adversarial Example Detection

The applications of deep learning are widely used across various fields. Additionally, Explainable AI (XAI) helps interpret model predictions, making them more reliable and trustworthy. Taking advantage of XAI techniques, we propose adopting decision logic from explanations to detect adversarial examples. We show that there exists a different interpretation distribution between normal and adversarial examples, as well as diverse decision logic of networks to distinguish them. Specifically, we first use DeepSHAP values to calculate the neuron contribution of the classification model layer-by-layer and show how to utilize this layer-wise explanation to distinguish between normal and adversarial examples. Second, we select critical neurons according to their SHAP values, generate a bitmap to represent the decision logic that interprets distribution of critical neurons, and propose a new approach that uses the decision logic instead of SHAP signature to detect adversarial examples. The preliminary results against the CIFAR-10 dataset demonstrate that the more SHAP layer information is given, the better accuracy can be achieved. Furthermore, using decision logic that concentrates on critical neurons can 1) outperform single or two layer SHAP signature detection approach, and 2) achieve competitive accuracy to all-layer SHAP signature detection with less resource requirement. We also show that the effectiveness of the proposed approach can be transferred to detect untrained attacks.