Reviewerly/exHarmony huggingface dataset:
Causalainer: Causal Explainer for Automatic Video Summarization

The goal of video summarization is to automatically shorten videos such that it conveys the overall story without losing relevant information. In many application scenarios, improper video summarization can have a large impact. For example in forensics, the quality of the generated video summary will affect an investigator’s judgment while in journalism it might yield undesired bias. Because of this, modeling explainability is a key concern. One of the best ways to address the explainability challenge is to uncover the causal relations that steer the process and lead to the result. Current machine learning-based video summarization algorithms learn optimal parameters but do not uncover causal relationships. Hence, they suffer from a relative lack of explainability. In this work, a Causal Explainer, dubbed Causalainer, is proposed to address this issue. Multiple meaningful random variables and their joint distributions are introduced to characterize the behaviors of key components in the problem of video summarization. In addition, helper distributions are introduced to enhance the effectiveness of model training. In visual-textual input scenarios, the extra input can decrease the model performance. A causal semantics extractor is designed to tackle this issue by effectively distilling the mutual information from the visual and textual inputs. Experimental results on commonly used benchmarks demonstrate that the proposed method achieves state-of-the-art performance while being more explainable.
OWL (Observe, Watch, Listen): Audiovisual Temporal Context for Localizing Actions in Egocentric Videos

Egocentric videos capture sequences of human activities from a first-person perspective and can provide rich multi-modal signals. However, most current localization methods use third-person videos and only incorporate visual information. In this work, we take a deep look into the effectiveness of audiovisual context in detecting actions in egocentric videos and introduce a simple-yet-effective approach via Observing, Watching, and Listening (OWL). OWL leverages audiovisual information and context for egocentric Temporal Action Localization (TAL). We validate our approach in two large-scale datasets, EPIC-KITCHENS and HOMAGE. Extensive experiments demonstrate the relevance of the audiovisual temporal context. Namely, we boost the localization performance (mAP) over visual-only models by +2.23% and +3.35% in the above datasets.
Incorporating Visual Grounding In GCN For Zero-shot Learning Of Human Object Interaction Actions

GCN-based zero-shot learning approaches commonly use fixed input graphs representing external knowledge that usually comes from language. However, such input graphs fail to incorporate the visual domain nuances. We introduce a method to ground the external knowledge graph visually. The method is demonstrated on a novel concept of grouping actions according to a shared notion and shown to be of superior performance in zero-shot action recognition on two challenging human manipulation action datasets, the EPIC Kitchens dataset, and the Charades dataset. We further show that visually grounding the knowledge graph enhances the performance of GCNs when an adversarial attack corrupts the input graph.
Cross-View Hierarchy Network for Stereo Image Super-Resolution

Stereo image super-resolution aims to improve the quality of high-resolution stereo image pairs by exploiting complementary information across views. To attain superior performance, many methods have prioritized designing complex modules to fuse similar information across views, yet overlooking the importance of intra-view information for high-resolution reconstruction. It also leads to problems of wrong texture in recovered images. To address this issue, we explore the interdependencies between various hierarchies from intra-view and propose a novel method, named Cross-View Hierarchy Network for Stereo Image Super-Resolution (CVHSSR). Specifically, we design a cross-hierarchy information mining block (CHIMB) that leverages channel attention and large kernel convolution attention to extract both global and local features from the intra-view, enabling the efficient restoration of accurate texture details. Additionally, a cross-view interaction module (CVIM) is proposed to fuse similar features from different views by utilizing cross-view attention mechanisms, effectively adapting to the binocular scene. Extensive experiments demonstrate the effectiveness of our method. CVHSSR achieves the best stereo image super-resolution performance than other state-of-the-art methods while using fewer parameters. The source code and pre-trained models are available at https://github.com/AlexZou14/CVHSSR.
OO-dMVMT: A Deep Multi-view Multi-task Classification Framework for Real-time 3D Hand Gesture Classification and Segmentation

Continuous mid-air hand gesture recognition based on captured hand pose streams is fundamental for human-computer interaction, particularly in AR / VR. However, many of the methods proposed to recognize heterogeneous hand gestures are tested only on the classification task, and the real-time low-latency gesture segmentation in a continuous stream is not well addressed in the literature. For this task, we propose the On-Off deep Multi-View Multi-Task paradigm (OO-dMVMT). The idea is to exploit multiple time-local views related to hand pose and movement to generate rich gesture descriptions, along with using heterogeneous tasks to achieve high accuracy. OO-dMVMT extends the classical MVMT paradigm, where all of the multiple tasks have to be active at each time, by allowing specific tasks to switch on/off depending on whether they can apply to the input. We show that OO-dMVMT defines the new SotA on continuous/online 3D skeleton-based gesture recognition in terms of gesture classification accuracy, segmentation accuracy, false positives, and decision latency while maintaining real-time operation.
Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective

Although fast adversarial training provides an efficient approach for building robust networks, it may suffer from a serious problem known as catastrophic overfitting (CO), where multi-step robust accuracy suddenly collapses to zero. In this paper, we for the first time decouple single-step adversarial examples into data-information and self-information, which reveals an interesting phenomenon called "self-fitting". Self-fitting, i.e., the network learns the self-information embedded in single-step perturbations, naturally leads to the occurrence of CO. When self-fitting occurs, the network experiences an obvious "channel differentiation" phenomenon that some convolution channels accounting for recognizing self-information become dominant, while others for data-information are suppressed. In this way, the network can only recognize images with sufficient self-information and loses generalization ability to other types of data. Based on self-fitting, we provide new insights into the existing methods to mitigate CO and extend CO to multi-step adversarial training. Our findings reveal a self-learning mechanism in adversarial training and open up new perspectives for suppressing different kinds of information to mitigate CO.
Q-Learning Algorithm

This chapter details the operation of the Q-Learning algorithm, one of the most widely used in algorithms Reinforcement Learning. The components of the algorithm and its demonstration through pseudocode are presented. Then, it is explained in detail how the algorithm works, illustrated with a visual example of an agent interacting in an environment, from the initialization of the Q-Table to the agent’s decision-making based on its experiences in the environment and the construction of a policy to be followed.
Sequential Estimation of Gaussian Process-Based Deep State-Space Models

We consider the problem of sequential estimation of the unknowns of state-space and deep state-space models that include estimation of functions and latent processes of the models. The proposed approach relies on Gaussian and deep Gaussian processes that are implemented via random feature-based Gaussian processes. In these models, we have two sets of unknowns, highly nonlinear unknowns (the values of the latent processes) and conditionally linear unknowns (the constant parameters of the random feature-based Gaussian processes). We present a method based on particle filtering where the parameters of the random feature-based Gaussian processes are integrated out in obtaining the predictive density of the states and do not need particles. We also propose an ensemble version of the method, with each member of the ensemble having its own set of features. With several experiments, we show that the method can track the latent processes up to a scale and rotation.
An Attack to One-Tap Authentication Services in Cellular Networks

The One-Tap Authentication (OTAuth) based on the cellular network is a password-less login service provided by Mobile Network Operator (MNO) through the unique communication gateway access technique. The service allows app users to quickly sign up or log in with their mobile phone numbers without entering a password. Due to its convenience, OTAuth has been widely used by various apps. However, some studies have elaborated that OTAuth services are of great drawbacks from the perspective of mobile security and identified several flawed designs, which make the MNO cannot distinguish malicious apps from normal ones and cause impersonation attacks. In this paper, we further analyze OTAuth services from the perspective of 4G and 5G cellular networks and focus on two important procedures in which the cellular network plays an important role in OTAuth services. Not surprisingly, we discover a new fundamental design flaw in determining whether the runtime environment supports OTAuth services. Moreover, we propose a mature attack paradigm by exploiting this flaw, which allows an attacker to login or register one app as a victim. To evaluate the impact of the attack, we have examined 100/90/100 Android/iOS/HarmonyOS apps for OTAuth services of 3 main-stream MNOs in China. The experimental results show that our proposed attack is applicable to almost all the apps that support OTAuth services, and affects more apps than the attacks that have been reported before. Finally, we propose several counter-measures to defend against the attack. Note that, for security’s sake, we have already reported our findings to authorized parties and received their confirmations.
Super-Resolution of SAR Images With Speckle Noise Based on Combination of Cubature Kalman Filter and Low-Rank Approximation

In this paper, two novel methods for SAR image super-resolution (SR) are proposed. The main challenge for SAR image SR reconstruction is speckle noise. For this reason, a novel algorithm termed the importance sampling cubature Kalman filter (ISCKF) is proposed to reconstruct a high resolution (HR) image from a series of low-resolution (LR) images. However, as the reconstructed image usually embraces residual noise visually, we establish a nonlinear low-rank optimization model in order to further reduce the speckle noise drastically. Correspondingly, an alternating direction method of multipliers based on the low-rank model (ADMM-LR) algorithm is proposed to solve it, which yields the other novel method termed ISCKF+ADMM-LR for SAR image SR. In addition, we establish the computational complexity of the proposed algorithms. The experimental results of both the simulated images and real SAR images demonstrate that the performance of the proposed methods are superior to some state-of-art methods in both SR and despeckle.
Reconstruction of Broken Writing Strokes in Greek Papyri

In the recent years, there has been an increased trend to digitize the historical manuscripts. This, in addition to preservation of these valuable collections, also allows public access to the digitized versions thus providing opportunities for researchers in pattern classification to develop computerized techniques for various applications. A common pre-processing step in such applications is the restoration of missing or broken strokes and makes the subject of our current study. More specifically, we work on isolated Greek characters extracted from handwriting on papyrus and employ a denoising auto-encoder to reconstruct the missing parts of characters. Performance evaluation using multiple evaluation metrics reports promising results.
Absformer: Transformer-Based Model for Unsupervised Multi-Document Abstractive Summarization

Multi-document summarization (MDS) refers to the task of summarizing the text in multiple documents into a concise summary. Abstractive MDS aims to generate a coherent and fluent summary for multiple documents using natural language generation techniques. In this paper, we consider the unsupervised abstractive MDS setting where there are only documents with no ground truth summaries provided, and we propose Absformer, a new Transformer-based method for unsupervised abstractive summary generation. Our method consists of a first step where we pretrain a Transformer-based encoder using the masked language modeling (MLM) objective as the pretraining task in order to cluster the documents into groups with semantically similar documents; and a second step where we train a Transformer-based decoder to generate abstractive summaries for the clusters of documents. To our knowledge, we are the first to successfully incorporate a Transformer-based model to solve the unsupervised abstractive MDS task. We evaluate our approach using three real-world datasets, and we demonstrate substantial improvements in terms of evaluation metrics over state-of-the-art abstractive-based unsupervised methods.
Crosslingual Handwritten Text Generation Using GANs

Generative Adversarial Networks, also called GANs, have been able to produce realistic looking handwritten images. However, training a GAN to generate usable data for improving handwriting recognition is a chicken and egg problem. In a low-resource language scenario, it would be beneficial to have a method that can generate more labeled data. But training such a GAN requires an amount of data that would not be available in a low-resource setting. In this paper, we present our work in data augmentation with a GAN that is independent of language and can be used to generate handwritten images by learning a mapping between printed and handwritten text. Our method is able to leverage training data from a source language and generate handwriting in a different target language. We show that in scenarios with adequate amounts of target language data, similar improvements in WER can be made by augmenting with either synthetic handwritten or printed text. However, in low resource scenarios, our GAN generated handwriting improves recognition results by 5–10% absolute over the baseline and 3–5% absolute over adding rendered printed text.
Adaptasi Pustakawan dalam Menghadapi Kemajuan Teknologi Informasi di Perpustakaan Utsman bin Affan Universitas Muslim Indonesia

The purpose of this study was to determine the adaptations made by librarians in dealing with advances in information technology at the Uthman Bin Affan library at the Muslim University of Indonesia and the obstacles faced by librarians in dealing with advances in information technology at the Uthman Bin Affan library at the Indonesian Muslim University. This study uses a descriptive type of research with a qualitative approach. Data collection techniques used are through observation, interviews, and documentation. Sources of data used are primary data and secondary data. Data processing and analysis techniques are data reduction, data presentation, and verification. The results showed that the activities followed by librarians in adapting to advances in information technology at the Uthman Bin Affan library at the Muslim University of Indonesia were participating in seminars, workshops, conferences, and technical guidance related to the adaptation of librarians in the face of advances in information technology. The obstacles faced by librarians in dealing with advances in information technology are the lack of librarian initiative to participate in activities that support the adaptation of librarians in the face of advances in information technology, inadequate facilities such as internet networks, and the rapid development of information technology, making librarians overwhelmed in adapting to advances in information technology in the world. library.
Hesitant hierarchical T–S fuzzy system with fuzzily weighted recursive least square

Hierarchical fuzzy systems have been an increasingly prevalent research topic, however, most of them are with fixed structures. This could reduce the flexibility of the method in applications and make it difficult to build an elastic multilayer structure for different input information. Hesitant fuzzy set (HFS) consists of rich and flexible hesitant fuzzy elements, which have been widely developed and applied in recent years, and related basic concepts and theories have been constantly developed and improved. By using the flexible expressing capabilities of HFS, we constructively construct a new serial-based hierarchical fuzzy system with flexible and adaptive rules in each layer, named after hesitant hierarchical T–S fuzzy system (HHFS). In the general HHFS, lengths of HFS elements play an important role generating constructions of each layer and consequent parameters of rules are adjusted by fuzzily weighted recursive least square method. It is the first time to build a multilayer fuzzy system using HFS. To adapt HHFS into the application of maneuvering target tracking, another newly adapted one, time-sequential hesitant fuzzy set (TSHFS) is utilized. The function of the conventional HFS is still generating structures of subsystems, and the TSHFS is utilized to optimize parameters of consequent parts with the new proposed approach for fluctuated hesitant information under TSHFS environment. In HHFS, the rules of subsystems are adaptively changeable according to different inputs, namely subsystems of each layer are with variant structures. Compared with state-of-the-art algorithms, the effectiveness and advantage of HHFS are exhibited in various experiments.
Beyond vanilla: Improved autoencoder-based ensemble in-vehicle intrusion detection system

Modern automobiles are equipped with a large number of electronic control units (ECUs) to provide safe, driver assistance and comfortable services. The controller area network (CAN) provides near real-time data transmission between ECUs with adequate reliability for in-vehicle communication. However, the lack of security measures such as authentication and encryption makes the CAN bus vulnerable to cyberattacks, which affect the safety of passengers and the surrounding environment. Detecting attacks on the CAN bus, particularly masquerade attacks, presents significant challenges. It necessitates an intrusion detection system (IDS) that effectively utilizes both CAN ID and payload data to ensure thorough detection and protection against a wide range of attacks, all while operating within the constraints of limited computing resources. This paper introduces an ensemble IDS that combines a gated recurrent unit (GRU) network and a novel autoencoder (AE) model to identify cyberattacks on the CAN bus. AEs are expected to produce higher reconstruction errors for anomalous inputs, making them suitable for anomaly detection. However, vanilla AE models often suffer from overgeneralization, reconstructing anomalies without significant errors, resulting in many false negatives. To address this issue, this paper proposes a novel AE called Latent AE, which incorporates a shallow AE into the latent space. The Latent AE model utilizes Cramér's statistic-based feature selection technique and a transformed CAN payload data structure to enhance its efficiency. The proposed ensemble IDS enhances attack detection capabilities by leveraging the best capabilities of independent GRU and Latent AE models, while mitigating the weaknesses associated with each individual model. The evaluation of the IDS on two public datasets, encompassing 13 different attacks, including sophisticated masquerade attacks, demonstrates its superiority over baseline models with near real-time detection latency of 25ms.
systematic literature review on untargeted model poisoning attacks and defense mechanisms in federated learning

In the past few years, Federated Learning has offered an optimistic solution to the privacy concerns of users who use different Machine Learning Models. But there are risks of exploiting the models by inside and outside adversaries. To preserve the data privacy and the model integrity, the Federated Learning model needs to be protected against the attackers. For this, the untargeted model poisoning attack where the model quality is compromised, needs to be detected early. This study focuses on finding various attack, detection and defense mechanisms against untargeted model poisoning attacks. Total 245 studies were found after searching Google Scholar, ScienceDirect and Scopus. After passing the selection criteria, only 15 studies were included in this systematic literature review. We have highlighted the attacks and defense mechanisms found in the related studies. Additionally, further study avenues in the area were recommended.
Post-selection in noisy Gaussian boson sampling: part is better than whole

Abstract Gaussian boson sampling (GBS) is originally proposed to show quantum advantage with quantum linear optical elements. Recently, several experimental breakthroughs based on GBS pointing to quantum computing supremacy have been presented. However, due to technical limitations, the outcomes of GBS devices are influenced severely by photon loss. Here, we present a practical method to reduce the negative effect caused by photon loss. We first show with explicit formulas that a GBS process can be mapped to another GBS processes. Based on this result, we propose a post-selection method which discards low-quality data according to our criterion to improve the performance of the final computational results, say part is better than whole. As an example, we show that the post-selection method can turn a GBS experiment that would otherwise fail in a ‘non-classicality test’ into one that can pass that test. Besides improving the robustness of computation results of current GBS devices, this post-selection method may also benefit the further development of GBS-based quantum algorithms.
Joint entity and relation extraction based on specific-relation attention mechanism and global features

In natural language processing, the research on extracting triple relations from unstructured texts to construct knowledge graphs has been widely concerned, however, few existing methods can solve the problem of overlapping triples and pay little attention to relations interaction with sentences. Besides, because entities have multiple forms, most methods rely on contextual features during entity recognition, and these models often misrepresent global relations. These all affect the accuracy of triplet extraction. To address the above issues, this paper proposes a joint extraction model with specific relation embeddings as well as incorporating global relations. It is called SRGF. Specifically, the model first converts the text into a vector representation that combines contextual features and global feature vectors through a pre-trained model and a graph convolutional network GCN, and then extracts topics. So entity extraction can be enhanced. An attention network guided by subject and relation information is employed to learn improved sentence representations for identifying the related objects. By doing so, it becomes possible to capture and model the interactions among entities, relations, and sentences, thus leading to better solutions for the problem of overlapping triples. Experimental results on public datasets NYT and WebNLG demonstrate that SRGF outperforms previous methods in triplet extraction.