Reviewerly/exHarmony huggingface dataset:
Spatial-Temporal Graphs for Cross-Modal Text2Video Retrieval

Cross-modal text to video retrieval aims to find relevant videos for given text queries, which is crucial for various real-world applications. The key to address this task is to build the correspondence between video and text such that the related samples from different modalities can be aligned. As the text (sentence) contains both nouns and verbs representing objects as well as their interactions, retrieving the relevant videos requires a fine-grained understanding of video contents---not only the semantic concepts (i.e., objects) but also the interactions between them. Nevertheless, the current approaches mostly represent videos with aggregated frame-level features for the learning of joint space and ignore the information of object interactions, which usually results in suboptimal retrieval performance. To improve the performance of cross-modal video retrieval, this paper proposes a framework that models videos as spatial-temporal graphs where nodes correspond to visual objects and edges correspond to the relations/interactions between objects. With the spatial-temporal graphs, the object interactions in frame sequences can be captured to enrich the video representations for joint space learning. Specifically, Graph Convolutional Network is introduced to learn the representations on spatial-temporal graphs, aiming to encode the spatial-temporal interactions between objects; while BERT is introduced to dynamically encode the sentence according to the context for cross-modal retrieval. Extensive experiments verify the effectiveness of the proposed framework and it achieves promising performances on both MSR-VTT and LSMDC datasets.
Financial Fraud Detection in Healthcare Using Machine Learning and Deep Learning Techniques

Healthcare sector is one of the prominent sectors in which a lot of data can be collected not only in terms of health but also in terms of finances. Major frauds happen in the healthcare sector due to the utilization of credit cards as the continuous enhancement of electronic payments, and credit card fraud monitoring has been a challenge in terms of financial condition to the different service providers. Hence, continuous enhancement is necessary for the system for detecting frauds. Various fraud scenarios happen continuously, which has a massive impact on financial losses. Many technologies such as phishing or virus-like Trojans are mostly used to collect sensitive information about credit cards and their owner details. Therefore, efficient technology should be there for identifying the different types of fraudulent conduct in credit cards. In this paper, various machine learning and deep learning approaches are used for detecting frauds in credit cards and different algorithms such as Naive Bayes, Logistic Regression, K-Nearest Neighbor (KNN), Random Forest, and the Sequential Convolutional Neural Network are skewed for training the other standard and abnormal features of transactions for detecting the frauds in credit cards. For evaluating the accuracy of the model, publicly available data are used. The different algorithm results visualized the accuracy as 96.1%, 94.8%, 95.89%, 97.58%, and 92.3%, corresponding to various methodologies such as Naive Bayes, Logistic Regression, K-Nearest Neighbor (KNN), Random Forest, and the Sequential Convolutional Neural Network, respectively. The comparative analysis visualized that the KNN algorithm generates better results than other approaches.
Explainable Artificial Intelligence for Tabular Data: A Survey

Machine learning techniques are increasingly gaining attention due to their widespread use in various disciplines across academia and industry.Despite their tremendous success, many such techniques suffer from the "black-box" problem, which refers to situations where the data analyst is unable to explain why such techniques arrive at certain decisions.This problem has fuelled interest in Explainable Artificial Intelligence (XAI), which refers to techniques that can easily be interpreted by humans.Unfortunately, many of these techniques are not suitable for tabular data, which is surprising given the importance and widespread use of tabular data in critical applications such as finance, healthcare, and criminal justice.Also surprising is the fact that, despite the vast literature on XAI, there are still no survey articles to date that focus on tabular data.Consequently, despite the existing survey articles that cover a wide range of XAI techniques, it remains challenging for researchers working on tabular data to go through all of these surveys and extract the techniques that are suitable for their analysis.Our article fills this gap by providing a comprehensive and up-to-date survey of the XAI techniques that are relevant to tabular data.Furthermore, we categorize the references covered in our survey, indicating the type of the model being explained, the approach being used to provide the explanation, and the XAI problem being addressed.Our article is the first to provide researchers with a map that helps them navigate the XAI literature in the context of tabular data.
“Shiny” crypto assets: A systemic look at gold-backed cryptocurrencies during the COVID-19 pandemic

In this paper, we empirically analyse the performance of five gold-backed stablecoins during the COVID-19 pandemic and compare them to gold, Bitcoin and Tether. In the digital assets' ecosystem, gold-backed cryptocurrencies have the potential to address regulatory and policy concerns by decreasing volatility of cryptocurrency prices and facilitating broader cryptocurrency adoption. We find that during the COVID-19 pandemic, gold-backed cryptocurrencies were susceptible to volatility transmitted from gold markets. Our results indicate that for the selected gold-backed cryptocurrencies, their volatility, and as a consequence, risks associated with volatility, remained comparable to the Bitcoin. In addition, gold-backed cryptocurrencies did not show safe-haven potential comparable to their underlying precious metal, gold.
Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detection

Monocular 3D object detection aims to localize 3D bounding boxes in an input single 2D image. It is a highly challenging problem and remains open, especially when no extra information (e.g., depth, lidar and/or multi-frames) can be leveraged in training and/or inference. This paper proposes a simple yet effective formulation for monocular 3D object detection without exploiting any extra information. It presents the MonoCon method which learns Monocular Contexts, as auxiliary tasks in training, to help monocular 3D object detection. The key idea is that with the annotated 3D bounding boxes of objects in an image, there is a rich set of well-posed projected 2D supervision signals available in training, such as the projected corner keypoints and their associated offset vectors with respect to the center of 2D bounding box, which should be exploited as auxiliary tasks in training. The proposed MonoCon is motivated by the Cramer–Wold theorem in measure theory at a high level. In implementation, it utilizes a very simple end-to-end design to justify the effectiveness of learning auxiliary monocular contexts, which consists of three components: a Deep Neural Network (DNN) based feature backbone, a number of regression head branches for learning the essential parameters used in the 3D bounding box prediction, and a number of regression head branches for learning auxiliary contexts. After training, the auxiliary context regression branches are discarded for better inference efficiency. In experiments, the proposed MonoCon is tested in the KITTI benchmark (car, pedestrian and cyclist). It outperforms all prior arts in the leaderboard on the car category and obtains comparable performance on pedestrian and cyclist in terms of accuracy. Thanks to the simple design, the proposed MonoCon method obtains the fastest inference speed with 38.7 fps in comparisons. Our code is released at https://git.io/MonoCon.
Transformer Feed-Forward Layers Are Key-Value Memories

Feed-forward layers constitute two-thirds of a transformer model's parameters, yet their role in the network remains under-explored.We show that feed-forward layers in transformerbased language models operate as key-value memories, where each key correlates with textual patterns in the training examples, and each value induces a distribution over the output vocabulary.Our experiments show that the learned patterns are human-interpretable, and that lower layers tend to capture shallow patterns, while upper layers learn more semantic ones.The values complement the keys' input patterns by inducing output distributions that concentrate probability mass on tokens likely to appear immediately after each pattern, particularly in the upper layers.Finally, we demonstrate that the output of a feed-forward layer is a composition of its memories, which is subsequently refined throughout the model's layers via residual connections to produce the final output distribution.
Model-Free Safety-Critical Control for Robotic Systems

This letter presents a framework for the safety-critical control of robotic systems, when safety is defined on safe regions in the configuration space. To maintain safety, we synthesize a safe velocity based on control barrier function theory without relying on a &#x2013; potentially complicated &#x2013; high-fidelity dynamical model of the robot. Then, we track the safe velocity with a tracking controller. This culminates in <i>model-free safety critical control</i>. We prove theoretical safety guarantees for the proposed method. Finally, we demonstrate that this approach is application-agnostic. We execute an obstacle avoidance task with a Segway in high-fidelity simulation, as well as with a Drone and a Quadruped in hardware experiments.
Deep Differentiable Random Forests for Age Estimation

Age estimation from facial images is typically cast as a label distribution learning or regression problem, since aging is a gradual progress. Its main challenge is the facial feature space w.r.t. ages is inhomogeneous, due to the large variation in facial appearance across different persons of the same age and the non-stationary property of aging. In this paper, we propose two Deep Differentiable Random Forests methods, Deep Label Distribution Learning Forest (DLDLF) and Deep Regression Forest (DRF), for age estimation. Both of them connect split nodes to the top layer of convolutional neural networks (CNNs) and deal with inhomogeneous data by jointly learning input-dependent data partitions at the split nodes and age distributions at the leaf nodes. This joint learning follows an alternating strategy: (1) Fixing the leaf nodes and optimizing the split nodes and the CNN parameters by Back-propagation; (2) Fixing the split nodes and optimizing the leaf nodes by Variational Bounding. Two Deterministic Annealing processes are introduced into the learning of the split and leaf nodes, respectively, to avoid poor local optima and obtain better estimates of tree parameters free of initial values. Experimental results show that DLDLF and DRF achieve state-of-the-art performance on three age estimation datasets.
AdapterDrop: On the Efficiency of Adapters in Transformers

Transformer models are expensive to fine-tune, slow for inference, and have large storage requirements.Recent approaches tackle these shortcomings by training smaller models, dynamically reducing the model size, and by training light-weight adapters.In this paper, we propose AdapterDrop, removing adapters from lower transformer layers during training and inference, which incorporates concepts from all three directions.We show that Adap-terDrop can dynamically reduce the computational overhead when performing inference over multiple tasks simultaneously, with minimal decrease in task performances.We further prune adapters from AdapterFusion, which improves the inference efficiency while maintaining the task performances entirely.
Assessment of Smart Home Assistants as an IoT

A smart home personal assistant technology is an intrinsic system, which incorporates many elements such as users, Smart Home Personal Assistants (SPA) devices, cloud, skill provider and other responsive devices. Even though Smart Home Personal Assistants give a robust security and privacy options, the devices face many weaknesses, which make the system vulnerable and can be comprised by adversaries, who can capitalize on limitations to gain access to delicate information and privacy of users. In this research, the aim is to assess how invention and innovation of security and SPA can be harnessed by users to interrelate with the system. Subsequently, this write-up will address both the problems related to the system and attempt to bring in solutions, which makes the technology more adaptable and versatile to all users. Initial studies show that some of the weakness underlying the technology include the open-nature of the voice channel, complexity of the architecture, software implications, and the utility of the technology to less proficient users. As a result, the study anticipates at solving the voice squatting attack, using the SPA intelligent assistant, incorporating a filer to filter the ultrasonic attack and noise as well as trying to assess the efficacy of the elements developed against the voice squatting attacks. The study found out that there is a need to mitigate the attacks on the blockchain technology and Natural Language Processors (NLP) to assure protection of SPA from attacks.
Episodic Transformer for Vision-and-Language Navigation

Interaction and navigation defined by natural language instructions in dynamic environments pose significant challenges for neural agents. This paper focuses on addressing two challenges: handling long sequence of subtasks, and understanding complex human instructions. We propose Episodic Transformer (E.T.), a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions. To improve training, we leverage synthetic instructions as an intermediate representation that decouples understanding the visual appearance of an environment from the variations of natural language instructions. We demonstrate that encoding the history with a transformer is critical to solve compositional tasks, and that pretraining and joint training with synthetic instructions further improve the performance. Our approach sets a new state of the art on the challenging ALFRED benchmark, achieving 38.4% and 8.5% task success rates on seen and unseen test splits.
A new SNOW stream cipher called SNOW-V

In this paper we are proposing a new member in the SNOW family of stream ciphers, called SNOW-V. The motivation is to meet an industry demand of very high speed encryption in a virtualized environment, something that can be expected to be relevant in a future 5G mobile communication system. We are revising the SNOW 3G architecture to be competitive in such a pure software environment, making use of both existing acceleration instructions for the AES encryption round function as well as the ability of modern CPUs to handle large vectors of integers (e.g. SIMD instructions). We have kept the general design from SNOW 3G, in terms of linear feedback shift register (LFSR) and Finite State Machine (FSM), but both entities are updated to better align with vectorized implementations. The LFSR part is new and operates 8 times the speed of the FSM. We have furthermore increased the total state size by using 128-bit registers in the FSM, we use the full AES encryption round function in the FSM update, and, finally, the initialization phase includes a masking with key bits at its end. The result is an algorithm generally much faster than AES-256 and with expected security not worse than AES-256.
Graphia: A platform for the graph-based visualisation and analysis of high dimensional data

Graphia is an open-source platform created for the graph-based analysis of the huge amounts of quantitative and qualitative data currently being generated from the study of genomes, genes, proteins metabolites and cells. Core to Graphia’s functionality is support for the calculation of correlation matrices from any tabular matrix of continuous or discrete values, whereupon the software is designed to rapidly visualise the often very large graphs that result in 2D or 3D space. Following graph construction, an extensive range of measurement algorithms, routines for graph transformation, and options for the visualisation of node and edge attributes are available, for graph exploration and analysis. Combined, these provide a powerful solution for the interpretation of high-dimensional data from many sources, or data already in the form of a network or equivalent adjacency matrix. Several use cases of Graphia are described, to showcase its wide range of applications in the analysis biological data. Graphia runs on all major desktop operating systems, is extensible through the deployment of plugins and is freely available to download from https://graphia.app/ .
Balanced Contrastive Learning for Long-Tailed Visual Recognition

Real-world data typically follow a long-tailed distribution, where a few majority categories occupy most of the data while most minority categories contain a limited number of samples. Classification models minimizing crossentropy struggle to represent and classify the tail classes. Although the problem of learning unbiased classifiers has been well studied, methods for representing imbalanced data are under-explored. In this paper, we focus on representation learning for imbalanced data. Recently, supervised contrastive learning has shown promising performance on balanced data recently. However, through our theoretical analysis, we find that for long-tailed data, it fails to form a regular simplex which is an ideal geometric configuration for representation learning. To correct the optimization behavior of SCL and further improve the performance of long-tailed visual recognition, we propose a novel loss for balanced contrastive learning (BCL). Compared with SCL, we have two improvements in BCL: classaveraging, which balances the gradient contribution of negative classes; class-complement, which allows all classes to appear in every mini-batch. The proposed balanced contrastive learning (BCL) method satisfies the condition of forming a regular simplex and assists the optimization of cross-entropy. Equipped with BCL, the proposed two-branch framework can obtain a stronger feature representation and achieve competitive performance on long-tailed benchmark datasets such as CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist2018.
Accurate and Scalable Cross-Architecture Cross-OS Binary Code Search with Emulation

Different from source code clone detection, clone detection (similar code search) in binary executables faces big challenges due to the gigantic differences in the syntax and the structure of binary code that result from different configurations of compilers, architectures and OSs. Existing studies have proposed different categories of features for detecting binary code clones, including CFG structures, n-gram in CFG, input/output values, etc. In our previous study and the tool BinGo, to mitigate the huge gaps in CFG structures due to different compilation scenarios, we propose a selective inlining technique to capture the complete function semantics by inlining relevant library and user-defined functions. However, only features of input/output values are considered in BinGo. In this study, we propose to incorporate features from different categories (e.g., structural features and high-level semantic features) for accuracy improvement and emulation for efficiency improvement. We empirically compare our tool, BinGo-E, with the pervious tool BinGo and the available state-of-the-art tools of binary code search in terms of search accuracy and performance. Results show that BinGo-E achieves significantly better accuracies than BinGo for cross-architecture matching, cross-OS matching, cross-compiler matching and intra-compiler matching. Additionally, in the new task of matching binaries of forked projects, BinGo-E also exhibits a better accuracy than the existing benchmark tool. Meanwhile, BinGo-E takes less time than BinGo during the process of matching.
Introduction to Voice Presentation Attack Detection and Recent Advances

Over the past few years, significant progress has been made in the field of presentation attack detection (PAD) for automatic speaker recognition (ASV). This includes the development of new speech corpora, standard evaluation protocols and advancements in front-end feature extraction and back-end classifiers. The use of standard databases and evaluation protocols has enabled for the first time the meaningful benchmarking of different PAD solutions. This chapter summarises the progress, with a focus on studies completed in the last 3 years. The article presents a summary of findings and lessons learned from two ASVspoof challenges, the first community-led benchmarking efforts. These show that ASV PAD remains an unsolved problem and that further attention is required to develop generalised PAD solutions which have potential to detect diverse and previously unseen spoofing attacks.
Image dehazing using Moore neighborhood-based gradient profile prior

Removing the haze from still images is a challenging issue. Dark Channel Prior (DCP) based dehazing techniques have been used to remove haze from still images. However, it produces poor results when image objects are inherently similar to the airlight and no shadow is cast on them. To eliminate this problem, a Moore neighborhood-based gradient profile prior is designed and developed to efficiently estimate the transmission map and atmospheric veil. The transmission map is also refined by developing a local activity-tuned anisotropic diffusion based filter. Afterward, image restoration is performed using the estimated transmission function. Thus, the proposed technique has an ability to remove haze from still images in an effective manner. The performance of the proposed technique is compared with recently developed seven dehazing techniques over synthetic and real-life hazy images. The experimental results depict the supremacy of the proposed technique in removing haze from still images when compared with several existing techniques. It also reveals that the restored image has little or no artifacts.
Detect or Track: Towards Cost-Effective Video Object Detection/Tracking

State-of-the-art object detectors and trackers are developing fast. Trackers are in general more efficient than detectors but bear the risk of drifting. A question is hence raised – how to improve the accuracy of video object detection/tracking by utilizing the existing detectors and trackers within a given time budget? A baseline is frame skipping – detecting every N-th frames and tracking for the frames in between. This baseline, however, is suboptimal since the detection frequency should depend on the tracking quality. To this end, we propose a scheduler network, which determines to detect or track at a certain frame, as a generalization of Siamese trackers. Although being light-weight and simple in structure, the scheduler network is more effective than the frame skipping baselines and flow-based approaches, as validated on ImageNet VID dataset in video object detection/tracking.