Reviewerly/exHarmony huggingface dataset:
“Blinks in the Dark”: Blink Estimation With Domain Adversarial Training (BEAT) Network

Blink detection plays an important role in many human-computer interaction applications for consumers.Unfortunately, deep neural network-based blink detection methods are not only susceptible to poor lighting conditions, but also the deep learning model is prone to bias due to the imbalance in the dataset distribution.To solve these problems, we propose Blink Estimation with Domain Adversarial Training (BEAT) network, which robustly detects blinks in unseen out-of-sample images captured even under poor lighting conditions by extracting domain-invariant features.BEAT network is inspired by the domain-adversarial neural network (DANN) but improved with several improvements including a lambda scheduler to stabilize adversarial training and a gradient decay layer to prevent the discriminative loss from overwhelming the classification loss.As a result, BEAT achieves faster and more accurate blink detection performances than other domain generalization methods for unseen target domains.In particular, BEAT's feature extractor model achieves state-of-the-art performance in terms of AUPR on popular benchmark datasets.Also, we suggest a practical optimal threshold for blink detection based on our insights gained from our experiments for consumer applications.
A Privacy-Preserving Symptoms Retrieval System with the Aid of Homomorphic Encryption and Private Set Intersection Schemes

This work presents an efficient and effective system allowing hospitals to share patients’ private information while ensuring that each hospital database’s medical records will not be leaked; moreover, the privacy of patients who access the data will also be protected. We assume that the thread model of the hospital’s security is semi-honest (i.e., curious but honest), and each hospital hired a trusted medical records department administrator to manage patients’ private information from other hospitals. With the help of Homomorphic Encryption- and Private Set Intersection -related algorithms, our proposed system protects patient privacy, allows physicians to obtain patient information across hospitals, and prevents threats such as troublesome insider attacks and man-in-the-middle attacks.
Modulating moderation: a history of objectionability in Twitter moderation practices

Abstract With their power to shape public discourse under unprecedented scrutiny, social media platforms have revamped their speech control practices in recent years by building complex systems of content moderation. The contours of this tectonic shift are relatively clear. Yet, little work has systematically documented, examined, and theorized this process. This article uses digital methods and web history to trace the evolution of objectionable content on Twitter content moderation policies and practices between 2006 and 2022. Its conclusions suggest that, more than abandoning an Americanized view of freedom of speech, Twitter has aimed at building a crisis-resistant speech architecture that can withstand external shocks, criticisms, and shifting speech norms. This kind of modulated moderation, as we term it, hinges on a form of normative plasticity, whose goal is not necessarily adjudicating content as more or less acceptable, but moderating it on the basis of evolving and ever contingent public conceptions of objectionability.
Towards Improvement of Grounded Cross-lingual Natural Language Inference with VisioTextual Attention

Natural Language Inference (NLI) has been one of the fundamental tasks in Natural Language Processing (NLP). Recognizing Textual Entailment (RTE) between the two pieces of text is a crucial problem. It adds further challenges when it involves two languages, i.e., in the cross-lingual scenario. In this paper, we propose VisioTextual-Attention (VTA) - an effective visual-textual coattention mechanism for multi-modal cross-lingual NLI. Through our research, we show that instead of using only linguistic input features, introducing visual features supporting the textual inputs improves the performance of NLI models if an effective cross-modal attention mechanism is carefully constructed. We perform several experiments on a standard cross-lingual textual entailment dataset in Hindi-English language pairs and show that the addition of visual information to the dataset along with our proposed VisioTextual Attention (VTA) enhances performance and surpasses the current state-of-the-art by 4.5%. Through monolingual experiments, we also show that the proposed VTA mechanism surpasses monolingual state-of-the-art by a margin of 2.89%. We argue that our VTA mechanism is model agnostic and can be used with other deep learning-based architectures for grounded cross-lingual NLI.
Regularizing Oversmoothing of Temporal Convolutional Networks for Action Segmentation into Human Assembly Operations

We investigated a (supervised) deep learning model for the automatic segmentation of manufacturing video data into a sequence of assembly operations. In the action segmentation for general human behavior, various temporal convolutional network (TCN)-based methods have been proposed and demonstrated to have stable performance using temporal features from captured extensive fields of image frames. However, they often make it difficult to detect unusual actions occurring in short durations, such as skipping assembly operations. In this paper, we address this drawback in the existing TCN model by introducing two techniques of differentiable group normalization and jumping knowledge to relax the oversmoothing effect. Then, we empirically show the performance of the proposed TCN variations using a benchmark for operational assembly work segmentation. We also present a preliminary result on irregular work detection.
Multi-graph multi-label learning with novel and missing labels

Real-life objects typically contain complex structures, and the graph is a prevalent presentation for describing such objects. Multi-graph multi-label (MGML) learning is a supervised learning framework for learning complicated objects associated with multiple labels, where each object is described as a bag-of-graphs representation. However, existing methods are dependent on transforming graphs into instances, which prevents the structure information from being fully utilized; in addition, the adaptability of MGML is insufficient for the assumptions that the label set is in a closed environment and complete. In real applications, some novel labels may exist outside the fixed set, and some values of labels are missing. Existing MGML approaches are unable to deal with these novel and missing labels. In this paper, we propose a new multi-graph multi-label learning with novel and missing labels (Mgmlnm) algorithm, which is the first attempt to learn novel and missing labels in the MGML problem. To preserve the structure information, we exploit the specific graph kernels to generate an efficient graph representation. To learn multi-graph data with novel and missing labels, we propose a unified objective function, which has a two-perspective regularization term, a projection-similarity regularization term, and a bag-dependent regularization term, enabling novel and missing labels to be modeled simultaneously and exploring the complex relationship between bag and graph labels. Two first-order proximity optimization methods are then developed to solve the unified problem efficiently. A theoretical analysis of the generalization bound is provided for the unified objective. Experimental results demonstrate the effectiveness of the proposed algorithm.
Explainable Human-Machine Teaming using Model Checking and Interpretable Machine Learning

The human-machine teaming paradigm promotes tight teamwork between humans and autonomous machines that collaborate in the same physical space. This paradigm is increasingly widespread in critical domains, such as healthcare and domestic assistance. These systems are expected to build a certain level of trust by enforcing dependability and exhibiting interpretable behavior. However, trustworthiness is negatively affected by the black-box nature of these systems, which typically make fully autonomous decisions that may be confusing for humans or cause hazards in critical domains. We present the EASE approach, whose goal is to build better trust in human-machine teaming leveraging statistical model checking and model-agnostic interpretable machine learning. We illustrate EASE through an example in healthcare featuring an infinite (dense) space of human-machine uncertain factors, such as diverse physical and physiological characteristics of the agents involved in the teamwork. Our evaluation demonstrates the suitability and cost-effectiveness of EASE in explaining dependability properties in human-machine teaming.
Actual Shape-Based Obstacle Avoidance Synthesized by Velocity–Acceleration Minimization for Redundant Manipulators: An Optimization Perspective

From the optimization perspective, this article proposes a novel actual shape-based obstacle avoidance synthesized by velocity–acceleration minimization (ASOA-VAM) scheme that performs operational tasks safely in a complex environment utilizing redundant manipulators. Concretely, an actual shape-based obstacle avoidance (ASOA) strategy with a variable magnitude escape acceleration using the Gilbert–Johnson–Keerthi distance algorithm is presented. Trajectory tracking, the end-effector’s errors feedback, and the joint multilevel physical limits (joint angle, -velocity, and -acceleration limits) avoidance are also incorporated into this optimization scheme. Meanwhile, the velocity–acceleration minimization (VAM) measure is developed. Combining the ASOA strategy with the VAM measure, the ASOA-VAM scheme is formed and further reformulated as a quadratic program (QP). Moreover, a recurrent neural network with theoretically provable convergence is designed to solve the QP online. Finally, simulations, comparisons, and experiments of a 7-degree-of-freedom manipulator with engineering applications illustrate the ASOA-VAM scheme’s effectiveness, accuracy, superiority, and physical realizability.
SFA-GAN: structure–frequency-aware generative adversarial network for underwater image enhancement

Influenced by light scattering, absorption and water impurities, the quality of underwater image is so poor that significant attention on underwater image enhancement (UIE) has been attracted for producing high-quality visuality. However, both supervised and unsupervised learning-based methods are either more demanding in practical application or easily trap in mapping ambiguity. To solve these issues, we propose a novel unsupervised learning-based underwater image enhancement framework, termed as SFA-GAN. Specifically, we first propose a structure–frequency-aware regularization, which can alleviate both structure and texture loss from spatial and frequency domain, and force SFA-GAN to better enhance underwater images. Afterward, unlike cycle-based underwater image enhancement model with two-sided learning, SFA-GAN adopts one-sided UIE, which is more efficient and faster than the widely used cycle-consistency architectures in training. Extensive experiments on datasets, including: EUVP, UFO-120 and UIEB, demonstrate that the proposed SFA-GAN can achieve state-of-art results on some metrics and produce more clear underwater images without sacrificing model complexity.
Considerations for Using Artificial Intelligence to Manage Authorized Push Payment (APP) Scams

Artificial Intelligence (AI)-based security intelligence modeling can be used to prevent, detect, and manage cyber threats. Data-driven AI solutions are currently undergoing rigorous research and design changes in their own field, but few scholars or practitioners frame authorized push payment (APP) scams as a unique cybersecurity concern, or tailor technical solutions based on local regulatory contexts. Drawing on a recent consultation publication by the UK Payment Systems Regulator on APP scams (November 2021), this article shows how AI can be leveraged to manage APP scams and explores some of the opportunities and risks one should consider when adopting such an approach. We highlight three scenarios: 1) Liability on payment service provider; 2) Liability on payor; and 3) Liability on payor with substantial public sector involvement. These examples illustrate how sociotechnical systems can play a design role, and consequently assist industry leaders and engineering management in prioritizing investment focus, strategic approaches, and technical solutions.
Dropout injection at test time for post hoc uncertainty quantification in neural networks

Among Bayesian methods, Monte Carlo dropout provides principled tools for evaluating the epistemic uncertainty of neural networks. Its popularity recently led to seminal works that proposed activating the dropout layers only during inference for evaluating epistemic uncertainty. This approach, which we call dropout injection, provides clear benefits over its traditional counterpart (which we call embedded dropout) since it allows one to obtain a post hoc uncertainty measure for any existing network previously trained without dropout, avoiding an additional, time-consuming training process. Unfortunately, no previous work thoroughly analyzed injected dropout and compared it with embedded dropout; therefore, we provide a first comprehensive investigation, focusing on regression problems. We show that the effectiveness of dropout injection strongly relies on a suitable scaling of the corresponding uncertainty measure, and propose an alternative method to implement it. We also considered the trade-off between negative log-likelihood and calibration error as a function of the scale factor. Experimental results on benchmark data sets from several regression tasks, including crowd counting, support our claim that dropout injection can effectively behave as a competitive post hoc alternative to embedded dropout.
SketchCADGAN: A generative approach for completing partially drawn query sketches of engineering shapes to enhance retrieval system performance

Retrieval systems are commonly used to find relevant data in large datasets. In engineering, these systems are useful for locating specific engineering shapes in a large dataset of engineering components. When end users want to search for a shape, they prefer a two-dimensional (2D) sketch over a three-dimensional (3D) object. However, users lacking domain knowledge may struggle to generate a complete query sketch and provide a partially completed sketch instead. Retrieving relevant information from partially drawn sketches is difficult because they may have missing edges, partially drawn circles, holes, ovals, etc. Most retrieval systems compare the similarity between the query and items in the database, so incomplete sketches may be ineffective in finding the relevant information. To address this problem with incomplete sketches, we propose a new generative adversarial network called SketchCADGAN. This network uses a two-stage cascaded architecture, with the first network attempting to predict a CAD model image from an incomplete sketch and the second network using the CAD model image to predict a completed sketch. Both networks are trained together adversarially. Our approach is proven more effective than other advanced techniques through qualitative and quantitative comparisons. Furthermore, we present the results of the retrieval system using both partially drawn and completed sketches, and demonstrate that incorporating completed sketches from the suggested cascaded GAN architecture results in improved retrieval performance.
Causal explanation for reinforcement learning: quantifying state and temporal importance

Explainability plays an increasingly important role in machine learning. Because reinforcement learning (RL) involves interactions between states and actions over time, it’s more challenging to explain an RL policy than supervised learning. Furthermore, humans view the world through a causal lens and thus prefer causal explanations over associational ones. Therefore, in this paper, we develop a causal explanation mechanism that quantifies the causal importance of states on actions and such importance over time. We also demonstrate the advantages of our mechanism over state-of-the-art associational methods in terms of RL policy explanation through a series of simulation studies, including crop irrigation, Blackjack, collision avoidance, and lunar lander.
One for more: Structured Multi-Modal Hashing for multiple multimedia retrieval tasks

Hashing has been widely studied in support of efficient multimedia retrieval due to its high computation and storage efficiency. However, existing multimedia hashing models are designed only for specific multimedia retrieval tasks, e.g., cross-modal retrieval and composite multi-modal retrieval. To simultaneously support multiple multimedia retrieval tasks, multiple different hashing models should be equipped in a multimedia retrieval system. This paper proposes a Structured Multi-Modal Hashing (SMMH) method, which can learn only a hash model for multiple multimedia retrieval tasks. Specifically, we generate structured hash codes by independently projecting the modality data into the Hamming space, where each part of hash codes preserves the specific characteristic of the specific modality data, and they are collaborated to represent the whole multi-modal data. To bridge the heterogeneous gap between multiple modalities, a fusion multi-modal space is constructed based on structured hash codes with label and pair-wise semantic supervision. Besides, we learn hash functions for composite multi-modal retrieval by projecting heterogeneous modalities into the collaborative multi-modal features. Finally, we develop a discrete hash optimization strategy to efficiently solve binary hash codes. Extensive experiments show that our model outperforms the strong baseline methods on multiple multimedia retrieval tasks. Source codes are available at https://github.com/ChaoqunZheng/SMMH.
Countering malicious content moderation evasion in online social networks: Simulation and detection of word camouflage

Content moderation is the process of screening and monitoring user-generated content online. It plays a crucial role in stopping content resulting from unacceptable behaviors such as hate speech, harassment, violence against specific groups, terrorism, racism, xenophobia, homophobia, or misogyny, to mention some few, in Online Social Platforms. These platforms make use of a plethora of tools to detect and manage malicious information; however, malicious actors also improve their skills, developing strategies to surpass these barriers and continuing to spread misleading information. Twisting and camouflaging keywords are among the most widely used techniques to evade platform content moderation systems. In response to this recent ongoing issue This paper presents an innovative approach to address this linguistic trend in social networks through the simulation of different content evasion techniques and a multilingual transformer model for content evasion detection. In this way a multilingual public tool Named "pyleetspeak" is shared with the scientific community, enabling the generation and simulation of content evasion through automatic word camouflage in a customizable way. Additionally a multilingual named-entity recognition (NER) transformer-based model is provided Designed for the recognition and detection of such evasion technique. The developed tool is multilingual Supporting over 20 languages (ar, az, da, de, el, en, es, fi, fr, hu, id, it, kk, nb, ne, nl, pt, ro, ru, sl, sv, tg, tr) and the NER model has been tested in English, Spanish, French, Italian, and German. This multilingual NER model is evaluated in different textual scenarios Detecting different types and mixtures of camouflage techniques Achieving an overall weighted F1 score of 0.8795. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content on social networks Making the fight against information disorders more effective
Deep Reinforcement Learning Applied to Multi-agent Informative Path Planning in Environmental Missions

Deep Reinforcement Learning algorithms have gained attention lately due to their ability to solve complex decision problems with a model-free and zero-derivative approach. In the case of multi-agent problems, these algorithms can help to easily find efficient cooperative policies in a feasible amount of time. In this chapter, we present the Informative Patrolling Problem, a commonplace task in the conservation of water resources. The approach is presented here as a convenient methodology for the synthesis of cooperative policies than can solve simultaneous objectives present in the unmanned monitoring of lakes and rivers: maximizing the collected information of water parameters and the collision-free routing with multiple surface vehicles. For this mixed objective, it is proposed a Deep Q-Learning scheme with a convolutional network as a shared fleet policy. In order to solve the credit assignment problem, it is proposed an effective multiagent decomposition of the informative reward with a discussion of other several state-of-the-art topics of Reinforcement Learning: noisy networks for enhanced exploration of the state-action domain, the use of a visual states, and the shaping of the reward function. This methodology, as it is quantitative demonstrated, allows a significant improvement in water resource monitoring compared to other heuristics.
Image encryption for Offshore wind power based on 2D-LCLM and Zhou Yi Eight Trigrams

Offshore wind power is an important part of the new power system, due to the complex and changing situation at ocean, its normal operation and maintenance cannot be done without information such as images, therefore, it is especially important to transmit the correct image in the process of information transmission. In this paper, we propose a new encryption algorithm for offshore wind power based on two-dimensional lagged complex logistic mapping (2D-LCLM) and Zhou Yi Eight Trigrams. Firstly, the initial value of the 2D-LCLM is constructed by the Sha-256 to associate the 2D-LCLM with the plaintext. Secondly, a new encryption rule is proposed from the Zhou Yi Eight Trigrams to obfuscate the pixel values and generate the round key. Then, 2D-LCLM is combined with the Zigzag to form an S-box. Finally, the simulation experiment of the algorithm is accomplished. The experimental results demonstrate that the algorithm can resistant common attacks and has prefect encryption performance.