Reviewerly/exHarmony huggingface dataset:
LNBFSM: A Food Safety Management System Using Blockchain and Lightning Network

Food safety management (FSM) has drawn attention recently, specifically after COVID-19 outbreak, due to lack of supervision in food products delivery chain. To address this issue, we propose a lightning network-blockchain FSM system. Our approach guarantees data authenticity by utilizing the technical characteristics of blockchain (e.g., tamper-resistant and traceability). We use lightning network to solve the throughput bottleneck issue in order to enable the proposed scheme adoptable in practice. The prototype of our approach has been evaluated and examined in our study.
Sparsity Turns Adversarial: Energy and Latency Attacks on Deep Neural Networks

Adversarial attacks have exposed serious vulnerabilities in deep neural networks (DNNs), causing misclassifications through human-imperceptible perturbations to DNN inputs. We explore a new direction in the field of adversarial attacks by suggesting attacks that aim to degrade the energy or latency of DNNs rather than their classification accuracy. As a specific embodiment of this new threat vector, we propose and demonstrate adversarial sparsity attacks, which modify a DNN's inputs so as to reduce sparsity (or the incidence of zeros) in its internal activation values. Exploiting sparsity in hardware and software has emerged as a popular approach to improve DNN efficiency in resource-constrained systems. The proposed attack, therefore, increases the execution time and energy consumption of sparsity-optimized DNN implementations, raising concern over their deployment in latency and energy-critical applications. We propose a systematic methodology to generate adversarial inputs for sparsity attacks by formulating an objective function that quantifies the network's activation sparsity and minimizing this function using iterative gradient-descent techniques. To prevent easy detection of the attack, we further ensure that the perturbation magnitude is within a specified constraint and that the perturbation does not affect classification accuracy. We launch both white-box and black-box versions of adversarial sparsity attacks on image recognition DNNs and demonstrate that they decrease activation sparsity by 1.16×-1.82×. On a sparsity-optimized DNN accelerator, the attack results in degradations of 1.12×-1.59× in latency and 1.18×-1.99× in energy-delay product (EDP). Additionally, we analyze the impact of various hyperparameters and constraints on the attack's efficacy. Finally, we evaluate defense techniques, such as activation thresholding and input quantization and demonstrate that the proposed attack is able to withstand them, highlighting the need for further efforts in this new direction within the field of adversarial machine learning.
Cognitive Modeling Based on Perceiving-Acting Cycle in Robotic Avatar System for Disabled Patients

In this study, we aim to develop a system of remote-controlled avatar robot for elderly and disabled patients. Most of teleoperation systems have interfaces to visually present the state of the robots including feedback information and receive the control commands manually sent from the operator. However, in elderly and patients with disabilities, they might have difficulty in the manual control of the robot. This paper therefore presents a multimodal interface for remotely controlling a robotic avatar. We furthermore propose a cognitive platform for remotely controlling a robot based on a concept of perceiving-acting cycle. The platform consists of a perceptual system for incremental environment modeling and an action system for extracting patterns of operator behavior. In each system, a self-organized neural network based on unsupervised learning is used. Moreover, we use a spiking neural network for spatial-temporal modeling of interaction between an operator and environment.
Input Dropout for Spatially Aligned Modalities

Computer vision datasets containing multiple modalities such as color, depth, and thermal properties are now commonly accessible and useful for solving a wide array of challenging tasks. However, deploying multi-sensor heads is not possible in many scenarios. As such many practical solutions tend to be based on simpler sensors, mostly for cost, simplicity and robustness considerations. In this work, we propose a training methodology to take advantage of these additional modalities available in datasets, even if they are not available at test time. By assuming that the modalities have a strong spatial correlation, we propose Input Dropout, a simple technique that consists in stochastic hiding of one or many input modalities at training time, while using only the canonical (e.g. RGB) modalities at test time. We demonstrate that Input Dropout trivially combines with existing deep convolutional architectures, and improves their performance on a wide range of computer vision tasks such as dehazing, 6-DOF object tracking, pedestrian detection and object classification.
On making physical the control of audio plugins

This paper reports the development of a prototype of smart musical instrument that uses a virtual analog audio plugin in conjunction with a dedicated tangible interface and a platform for embedded audio. The adopted design approach started from an analog synthesizer, passed from its digital emulation, and returned to the analog domain via the real-time, physical control of the digital synthesizer. The prototype can be considered as an instance of a class of musical devices that allow one to give physical form to the control of virtual analog software. We present an analysis of online sources that were retrieved following the release of the prototype at an international music trade show. Overall, results preliminary validate the concept underlying the development of the prototype and reveal its potential for both digital musical instruments development and use. Benefits of the proposed class of musical devices include a higher degree of control intimacy of a plugin compared to its use with conventional interfaces such as mice and screens of desktop computers, as well as the use of audio plugins in ubiquitous musical activities.
Low Cost Data Glove for Hand Gesture Recognition by Finger Bend Measurement

An approach for measuring local hand motions is explored by using induction coils as sensors for a data glove. The primary goal is to deduce finger joint angles in real time and recognize the corresponding hand gestures. The task is accomplished by relating the finger bending with the induced voltage measurements. The induced voltages are subsequently processed to extract the information of the finger angles. Induction coil properties are explored to realize their significance in finger bending measurement. The principle of electromagnetic induction is used to accomplish near field wireless transfer of energy between the coils. Different human hand motion constraints are also exploited to increase the effectiveness of the proposed approach. The technique is realized and tested by prototyping an embedded system that ensures easy comprehension of its working. Experimental results show the efficacy of the proposed design.
Complex Spatial-Temporal Attention Aggregation For Video Person Re-Identification

Video-based person re-identification (Re-ID) aims to match pedestrian tracklets of the same identity captured by different cameras. Existing works usually compute the video-level feature representation via simple frame-level feature aggregation, such as average pooling and max pooling. However, the performance of such methods degenerates severely under low signal-noise ratio and partial occlusions. In this paper, we propose a novel Complex Spatial-Temporal Attention Aggregation (CAA), which fully exploits the discriminative information in spatial-temporal dimension via the combination of two aggregation method, namely region-aware aggregation and region-regardless aggregation. We evaluate the proposed method in three widely used video Re-ID datasets, including MARS, iLIDS-VID, and PRID-2011. The experimental results demonstrate that the proposed method outperforms the state of the arts.
MARLeME: A Multi-Agent Reinforcement Learning Model Extraction Library

Multi-Agent Reinforcement Learning (MARL) encompasses a powerful class of methodologies that have been applied in a wide range of fields. An effective way to further empower these methodologies is to develop approaches and tools that could expand their interpretability and explainability. In this work, we introduce MARLeME: a MARL model extraction library, designed to improve explainability of MARL systems by approximating them with symbolic models. Symbolic models offer a high degree of interpretability, well-defined properties, and verifiable behaviour. Consequently, they can be used to inspect and better understand the underlying MARL systems and corresponding MARL agents, as well as to replace all/some of the agents that are particularly safety and security critical. In this work, we demonstrate how MARLeME can be applied to two well-known case studies (Cooperative Navigation and RoboCup Takeaway), using extracted models based on Abstract Argumentation.
Neural CAPTCHA networks

To protect against attacks by malicious computer programs, many websites apply the CAPTCHA (short for completely automated public turing test to tell computers and humans apart) technique for security protection. The distortion, rotation and deformation of the characters or puzzles in CAPTCHAs increase the difficulty for machines to automatically recognize them. State-of-the-art CAPTCHA recognition algorithms generally use convolutional neural networks (CNNs) without considering the spatially sequential property of the characters/image features. To address this problem, we propose a new CAPTCHA recognition algorithm called neural CAPTCHA networks (NCNs). NCNs use a convolutional structure to extract CAPTCHA image features, and use bidirectional recurrent modules to learn the spatially sequential information in CAPTCHAs. We have applied NCNs to recognize text-based CAPTCHAs, including arithmetic operation, character recognition and character matching CAPTCHAs, and puzzle-based CAPTCHAs. For arithmetic operation and character recognition CAPTCHAs, we obtained 100% accuracy on the SOIEC CAPTCHA dataset, for the character matching task, we obtained 99.3% accuracy on the SOIEC CAPTCHA dataset, while for the puzzle-based CAPTCHAs, we obtained 98.13% accuracy. These experimental results demonstrate the advantages of NCNs over related state-of-the-art approaches for CAPTCHA recognition.
Reflections on Work Instructions of Assembly Tasks

For assembly workers, the quality of their work instructions is essential. A high quality instruction can significantly contribute to work efficiency and quality of the result. However, oftentimes it is not obvious what aspects of an instruction make it well applicable for the respective setting. This paper focuses on the uncovering of shortcomings and challenges in work instructions. To this end, we conducted fieldwork in the day-to-day work of assembly workers in the shop floor environment of manufacturing companies. Different empirical methods were used to gather data for analysis. Based on the findings, an interactive prototype built upon real-world instructions was developed and evaluated iteratively using a multiple-case study. The results discuss key findings concerning the interaction with, and usability and user experience of digital work instructions.
Research on Causes of Transmission Line Fault Based on Decision Tree Classification

Aiming at the needs of transmission line fault analysis and line inspection and maintenance, mainly considering the severe weather factors, in order to improve the analysis speed and inspection efficiency, this paper collected multi-source line data, established a line attribute information model, and proposed a decision tree-based algorithm for identifying the cause of line fault, the rules extracted from the historical abnormal data of the line establish a decision tree for line fault analysis. The analysis of test samples in the case proves the effectiveness of the algorithm model and the high accuracy of fault cause identification. It provides effective artificial intelligence diagnostic support for operation, maintenance, inspection and repair.
Reality as a simulation of reality: robot illusions, fundamental limits, and a physical demonstration

We consider problems in which robots conspire to present a view of the world that differs from reality. The inquiry is motivated by the problem of validating robot behavior physically despite there being a discrepancy between the robots we have at hand and those we wish to study, or the environment for testing that is available versus that which is desired, or other potential mismatches in this vein. After formulating the concept of a convincing illusion, essentially a notion of system simulation that takes place in the real world, we examine the implications of this type of simulability in terms of infrastructure requirements. Time is one important resource: some robots may be able to simulate some others but, perhaps, only at a rate that is slower than real-time. This difference gives a way of relating the simulating and the simulated systems in a form that is relative. We establish some theorems, including one with the flavor of an impossibility result, and providing several examples throughout. Finally, we present data from a simple multi-robot experiment based on this theory, with a robot navigating amid an unbounded field of obstacles."Truth is beautiful, without doubt; but so are lies."-Ralph Waldo Emerson.
Context-Aware Synergetic Multiplex Network for Multi-organ Segmentation of Cervical Cancer MRI

Generative Adversarial Networks (GANs) have increasingly broken records in solving challenging medical image analyses problems such as medical image de-noising, segmentation, detection, classification or reconstruction. However, to the best of our knowledge, they have not been used for female pelvic multi-organ segmentation. Accurate segmentation of uterine cervical cancer (UCC) organs (i.e., bladder, vagina and tumor) from magnetic resonance imaging (MRI) is crucial for effective UCC staging. However, it is a highly challenging task due to 1) noisy MR images, 2) within-subject large variability in structure and intensity of UCC organs, and 3) across-subject variability. More importantly, there have been very limited works on how to aggregate different interactions across MRI views using GANs for multi-organ segmentation while providing context information. In this work, we propose a novel synergetic multiplex network (SMN) using multi-stage deep learning architecture based on cycle-GAN to segment pelvic multi-organ using complementary multi-view MRI, introducing three major contributions in multi-organ segmentation literature: (1) Modeling the interactions across data views using a novel multiplex architecture composed of multiple layers. Each SMN layer nests a cascade of view-specific context-aware cycle-GANs and synergistically communicates context information to other paralleled view-specific layers via multiplex coupling links. (2) SMN captures shared and complementary information between different views to segment UCC in different MRI views. (3) It enforces the spatial consistency between neighboring pixels within the same tissue for UCC segmentation. Specifically, in a gradual and deep manner, the proposed method improves the segmentation results by iteratively providing more refined context information from other views to train the next segmentation cycle-GAN in the SMN layer. We evaluated our SMN framework using 15 T2w-MR sequences with axial and sagittal views. We show that SMN is robust for the UCC segmentation task by significantly (\(p<0.05\)) outperforming comparison segmentation methods.
Toward Tag-free Aspect Based Sentiment Analysis: A Multiple Attention Network Approach

Existing aspect based sentiment analysis (ABSA) approaches leverage various neural network models to extract the aspect sentiments via learning aspect-specific feature representations. However, these approaches heavily rely on manual tagging of user reviews according to the predefined aspects as the input, a laborious and time-consuming process. Moreover, the underlying methods do not explain how and why the opposing aspect level polarities in a user review lead to the overall polarity. In this paper, we tackle these two problems by designing and implementing a new Multiple-Attention Network (MAN) approach for more powerful ABSA without the need for aspect tags using two new tag-free data sets crawled directly from TripAdvisor (https://www.tripadvisor.com). With the Self- and Position-Aware attention mechanism, MAN is capable of extracting both aspect level and overall sentiments from the text reviews using the aspect level and overall customer ratings, and it can also detect the vital aspect(s) leading to the overall sentiment polarity among different aspects via a new aspect ranking scheme. We carry out extensive experiments to demonstrate the strong performance of MAN compared to other state-of-the-art ABSA approaches and the explainability of our approach by visualizing and interpreting attention weights in case studies.
The Film Industry Leaps into Artificial Intelligence: Scope and Challenges by the Filmmakers

Artificial Intelligence (AI) is usually associated with assisting humans with tasks that are either ordinary or can be done better with automation. Since then technology grew, representation of AI and Robotics also evolved in films, especially the robots, which has given surplus roles from the antagonist, protagonist, the romantic, to a deadly weapon capable of machine learning. With the advent of CGI in the 2000s, an AI flareup took place in the world of cinema and came up with worthy concepts of AI-based movies. Today, the AI system can robotically deliver advanced visual effects which makes it easier for performers to act their alternative characters and creatures. Data Analytics and AI technology have already made their debut in films and are going to play a bigger role in the filmmaking process. Till now, films were made on AI. In the future, AI will be writing films, robots will perform, and animation will be done by a deep learning algorithm process. The study aimed to observe the AI technologies used in the film industry. The methods adopted to achieve the goals of the study is the content analysis method. The study will conclude that technology is constantly improving, and AI can become an inescapable force that could renovate the films and take it to the next level.
FedExg: Federated Learning with Model Exchange

Federated learning has successfully mitigated the privacy concerns for deep learning over distributed data, but suffers from performance decay led by the non-identical distribution of data. In this paper, we propose a novel framework, namely FedExg, to improve federated learning for non-IID data with model exchange. Two strategies, i.e., FedExg-S and FedExg-A, are developed to realize model exchange in a privacy-preserving fashion. FedExg-S randomly shuffles the models for broadcasting, while FedExg-A improves FedExg-S with partial aggregation to avoid model inversion attack. Theoretical analysis demonstrates that FedExg achieves a reduced upper bound of average regret in comparison to traditional model averaging methods. Experimental results show that FedExg improves image classification tasks on MNIST and CIFAR-10 dataset.
Establishing Secure Communication Channels Using Remote Attestation with TPM 2.0

Remote attestation allows a verifier to remotely check the integrity of a trusted computing platform. In recent years a number of attestation protocols based on Trusted Platform Modules (TPMs) have been proposed. These protocols cryptographically verify a trusted platform’s state by exchanging TPM-signed quotes. Some of them also establish an encrypted channel to the trusted platform, which allows the verifier to transmit information that only the attested software stack can read. However, many existing attestation protocols are either vulnerable against man-in-the-middle attacks, or depend on outdated TPM specifications. In this work we analyze a recently developed attestation protocol that is being actively used to interconnect highly distributed trusted applications. We find this protocol to be vulnerable against a variant of the classical relay attack. In response to this threat we develop a lightweight remote attestation protocol based on the TPM 2.0 specification that is not vulnerable to this attack. Unlike previous proposals, our protocol relies solely on the TPM to establish a shared key on the attested channel, which significantly reduces its attack surface. Our protocol supports mutual attestation, perfect forward secrecy and is independent of the underlying network stack. We provide a reference implementation of our protocol and compare its performance to previous proposals. We also analyze its security properties using the Tamarin theorem prover.
Deep Disentangled Hashing with Momentum Triplets for Neuroimage Search

Neuroimaging has been widely used in computer-aided clinical diagnosis and treatment, and the rapid increase of neuroimage repositories introduces great challenges for efficient neuroimage search. Existing image search methods often use triplet loss to capture high-order relationships between samples. However, we find that the traditional triplet loss is difficult to pull positive and negative sample pairs to make their Hamming distance discrepancies larger than a small fixed value. This may reduce the discriminative ability of learned hash code and degrade the performance of image search. To address this issue, in this work, we propose a deep disentangled momentum hashing (DDMH) framework for neuroimage search. Specifically, we first investigate the original triplet loss and find that this loss function can be determined by the inner product of hash code pairs. Accordingly, we disentangle hash code norms and hash code directions and analyze the role of each part. By decoupling the loss function from the hash code norm, we propose a unique disentangled triplet loss, which can effectively push positive and negative sample pairs by desired Hamming distance discrepancies for hash codes with different lengths. We further develop a momentum triplet strategy to address the problem of insufficient triplet samples caused by small batch-size for 3D neuroimages. With the proposed disentangled triplet loss and the momentum triplet strategy, we design an end-to-end trainable deep hashing framework for neuroimage search. Comprehensive empirical evidence on three neuroimage datasets shows that DDMH has better performance in neuroimage search compared to several state-of-the-art methods.