Reviewerly/exHarmony huggingface dataset:
Video Moment Localization via Deep Cross-Modal Hashing

Due to the continuous booming of surveillance and Web videos, video moment localization, as an important branch of video content analysis, has attracted wide attention from both industry and academia in recent years. It is, however, a non-trivial task due to the following challenges: temporal context modeling, intelligent moment candidate generation, as well as the necessary efficiency and scalability in practice. To address these impediments, we present a deep end-to-end cross-modal hashing network. To be specific, we first design a video encoder relying on a bidirectional temporal convolutional network to simultaneously generate moment candidates and learn their representations. Considering that the video encoder characterizes temporal contextual structures at multiple scales of time windows, we can thus obtain enhanced moment representations. As a counterpart, we design an independent query encoder towards user intention understanding. Thereafter, a cross-model hashing module is developed to project these two heterogeneous representations into a shared isomorphic Hamming space for compact hash code learning. After that, we can effectively estimate the relevance score of each “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">moment-query</i> ” pair via the Hamming distance. Besides effectiveness, our model is far more efficient and scalable since the hash codes of videos can be learned offline. Experimental results on real-world datasets have justified the superiority of our model over several state-of-the-art competitors.
RSG: A Simple but Effective Module for Learning Imbalanced Datasets

Imbalanced datasets widely exist in practice and are a great challenge for training deep neural models with a good generalization on infrequent classes. In this work, we propose a new rare-class sample generator (RSG) to solve this problem. RSG aims to generate some new samples for rare classes during training, and it has in particular the following advantages: (1) it is convenient to use and highly versatile, because it can be easily integrated into any kind of convolutional neural network, and it works well when combined with different loss functions, and (2) it is only used during the training phase, and therefore, no additional burden is imposed on deep neural networks during the testing phase. In extensive experimental evaluations, we verify the effectiveness of RSG. Furthermore, by leveraging RSG, we obtain competitive results on Imbalanced CIFAR and new state-of-the-art results on Places-LT, ImageNet-LT, and iNaturalist 2018. The source code is available at https://github.com/Jianf-Wang/RSG.
An evidential classifier based on Dempster-Shafer theory and deep learning

We propose a new classifier based on Dempster-Shafer (DS) theory and a convolutional neural network (CNN) architecture for set-valued classification. In this classifier, called the evidential deep-learning classifier, convolutional and pooling layers first extract high-dimensional features from input data. The features are then converted into mass functions and aggregated by Dempster's rule in a DS layer. Finally, an expected utility layer performs set-valued classification based on mass functions. We propose an end-to-end learning strategy for jointly updating the network parameters. Additionally, an approach for selecting partial multi-class acts is proposed. Experiments on image recognition, signal processing, and semantic-relationship classification tasks demonstrate that the proposed combination of deep CNN, DS layer, and expected utility layer makes it possible to improve classification accuracy and to make cautious decisions by assigning confusing patterns to multi-class sets.
Online Student Authentication and Proctoring System Based on Multimodal Biometrics Technology

Identity verification and proctoring of online students are one of the key challenges to online learning today. Especially for online certification and accreditation, the training organizations need to verify that the online students who completed the learning process and received the academic credits are those who registered for the courses. Furthermore, they need to ensure that these students complete all the activities of online training without cheating or inappropriate behaviours. The COVID-19 pandemic has accelerated (abruptly in certain cases) the migration and implementation of online education strategies and consequently the need for safe mechanisms to authenticate and proctor online students. Nowadays, there are several technologies with different grades of automation. In this paper, we deeply describe a specific solution based on the authentication of different biometric technologies and an automatic proctoring system (system workflow as well as AI algorithms), which incorporates features to solve the main concerns in the market: highly scalable, automatic, affordable, with few hardware and software requirements for the user, reliable and passive for the student. Finally, the technological performance test of the large scale system, the usability-privacy perception survey of the user and their results are discussed in this work.
RBT: A distributed reputation system for blockchain-based peer-to-peer energy trading with fairness consideration

It has been proved by recent works that reputation can help to improve the efficiency of blockchain and enhance the fairness of energy trading markets. However, the application of distributed reputation in the energy field has not been fully studied yet. In this paper, we design a distributed reputation system to simulate real-world trust in blockchain-based peer-to-peer energy trading. It is a comprehensive reputation system in the sense that reputation scores are decided by the behavior of participants as consensus nodes, energy buyer, and energy sellers. Its implementation relies on blockchain, especially the smart contract technology, to achieve distributed and automatic reputation management. The distributed reputation system in turn helps to implement a delegated consensus algorithm for blockchain and a reputation-based k-double auction matchmaking scheme for peer-to-peer energy trading. In addition, we define a fairness indicator to capture the reputation-based average benefits and costs when considering reputation as the contribution to the peer-to-peer energy trading market. By simulating the comprehensive system, the numerical results demonstrate the effects of distributed reputation in improving the efficiency of blockchain and balancing fairness indicators between sellers and buyers during peer-to-peer energy trading. As far as we know, this paper is one of the few works to provide a formal method of evaluating the fairness of the peer-to-peer energy trading market.
Automated blast cell detection for Acute Lymphoblastic Leukemia diagnosis

Acute Lymphoblastic Leukemia (ALL) is a cancer of the blood cells which is characterized by a large number of immature lymphocytes, known as blast cells (myeloblasts). To aid the ALL diagnosis, we propose to automate the blast cell detection using Artificial Intelligence (AI). Our automation system incorporates an object detection method that predicts leukemic cells from microscopic blood smear images. We have implemented version 4 of the You Only Look Once (YOLOv4) algorithm for both cell detection and cell classification. As such, the classification was set up as a binary problem, where each cell was labeled as either blast cells (ALL) or healthy cells (HEM). The Object Detection algorithm was trained and tested with images from the ALL_IDB1 and C_NMC_2019 dataset. The mAP (Mean Average Precision) was 96.06 % for the ALL-IDB1 dataset and 98.7 % for the C_NMC_2019 dataset. Both models were trained with Google Colaboratory using a Nvidia Tesla P-100 GPU. This proposed blast cell detection algorithm might be used as an adjunct tool during pre-screening where it can help to detect Leukemia based on microscopic blood smear images.
The disruption of blockchain in auditing – a systematic literature review and an agenda for future research

Purpose This paper presents a systematic literature review, including content and bibliometric analyses, of the impact of blockchain technology (BT) in auditing, to identify trends, research areas and construct an agenda for future research. Design/methodology/approach The authors include studies from 2010 to 2020 in their structured literature review (SLR), using accounting journals on the Scopus database, which yielded 40 articles with blockchain and auditing at its core. Findings One of the contributions of the authors’ analyses is to group the prior research, and therefore also the agenda for future research, into three main research areas: (1) Blockchain as a tool for auditing professionals to improve business information systems to save time and prevent fraud; (2) Smart contracts enabling Audit 4.0 efficiency, reporting, disclosure and transparency; (3) Cryptocurrency and initial coin offerings (ICOs) as a springboard for corporate governance and new venture financing. The authors’ findings have several important implications for practice and theory. Practical implications The results of this study emphasise that (1) the disruption of blockchain in auditing is in a nascent phase and there is a need for compelling empirical studies and potential for the involvement of practitioners; (2) there may be a need to reconsider audit procedures especially suited for digitalisation and BT adoption; (3) standards, guidelines and training are required to pivot towards and confront the challenge BT will represent for auditing; and (4) there are two sides to the BT coin for auditing, enthusiasm about the potential and risk upon implementation. These practical implications can also be seen as a template for future research in a quest to align theory and practice. Originality/value The authors’ SLR facilitates the identification of research areas and implications, forming a useful baseline for practitioners, professionals and academics, as they draft the state of the art on the disruption of blockchain in auditing, highlighting how BT is changing auditing activities and traditions.
Adaptive Class Suppression Loss for Long-Tail Object Detection

To address the problem of long-tail distribution for the large vocabulary object detection task, existing methods usually divide the whole categories into several groups and treat each group with different strategies. These methods bring the following two problems. One is the training inconsistency between adjacent categories of similar sizes, and the other is that the learned model is lack of discrimination for tail categories which are semantically similar to some of the head categories. In this paper, we devise a novel Adaptive Class Suppression Loss (ACSL) to effectively tackle the above problems and improve the detection performance of tail categories. Specifically, we introduce a statistic-free perspective to analyze the long-tail distribution, breaking the limitation of manual grouping. According to this perspective, our ACSL adjusts the suppression gradients for each sample of each class adaptively, ensuring the training consistency and boosting the discrimination for rare categories. Extensive experiments on long-tail datasets LVIS and Open Images show that the our ACSL achieves 5.18% and 5.2% improvements with ResNet50-FPN, and sets a new state of the art. Code and models are available at https://github.com/CASIA-IVA-Lab/ACSL.
Relational Message Passing for Knowledge Graph Completion

Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. In this work, we propose a relational message passing method for knowledge graph completion. Different from existing embedding-based methods, relational message passing only considers edge features (i.e., relation types) without entity IDs in the knowledge graph, and passes relational messages among edges iteratively to aggregate neighborhood information. Specifically, two kinds of neighborhood topology are modeled for a given entity pair under the relational message passing framework: (1) Relational context, which captures the relation types of edges adjacent to the given entity pair; (2) Relational paths, which characterize the relative position between the given two entities in the knowledge graph. The two message passing modules are combined together for relation prediction. Experimental results on knowledge graph benchmarks as well as our newly proposed dataset show that, our method PathCon outperforms state-of-the-art knowledge graph completion methods by a large margin. PathCon is also shown applicable to inductive settings where entities are not seen in training stage, and it is able to provide interpretable explanations for the predicted results. The code and all datasets are available at https://github.com/hwwang55/PathCon.
Blockchain technology for port logistics capability: Exclusive or sharing

Although the application of blockchain technology in maritime industry can improve the customs clearance efficiency and logistics transparency in the container cargo supply chains, it generates the unit operations cost and setup cost. In this study, a spatial model is proposed to conduct an investigation into the two heterogeneous ports when blockchain technology is applied under inter-port competition. Different utility-driven operations models are built to highlight the values of blockchain technology for port logistics capability, and to reveal the equilibria under the optimal strategies and equilibrium strategies. It is noticed that the relationship between the unit net benefit and unit operations cost brought out by blockchain technology is a key factor determining whether the blockchain technology is applicable. When the unit operations cost is lower than unit net benefit, and the setup cost is high, there is a win-win situation for two ports to apply traditional technology. It is further verified to be Pareto-optimal. In addition, three lose-lose situations are also identified, which are classic Prisoner's Dilemmas. Consequently, a horizontal collaboration mechanism “blockchain technology sharing + compensation” (BTSC) is designed to not only break these three Prisoner's Dilemmas, but also achieve the Pareto-optimum. Furthermore, we discuss some extensions to prove the robustness of our main qualitative insights.
Are GAN Generated Images Easy to Detect? A Critical Analysis of the State-Of-The-Art

The advent of deep learning has brought a significant improvement in the quality of generated media. However, with the increased level of photorealism, synthetic media are becoming hardly distinguishable from real ones, raising serious concerns about the spread of fake or manipulated information over the Internet. In this context, it is important to develop automated tools to reliably and timely detect synthetic media. In this work, we analyze the state-of-the-art methods for the detection of synthetic images, highlighting the key ingredients of the most successful approaches, and comparing their performance over existing generative architectures. We will devote special attention to realistic and challenging scenarios, like media uploaded on social networks or generated by new and unseen architectures, analyzing the impact of suitable augmentation and training strategies on the detectors’ generalization ability.
Privacy-Preserving Asynchronous Vertical Federated Learning Algorithms for Multiparty Collaborative Learning

The privacy-preserving federated learning for vertically partitioned (VP) data has shown promising results as the solution of the emerging multiparty joint modeling application, in which the data holders (such as government branches, private finance, and e-business companies) collaborate throughout the learning process rather than relying on a trusted third party to hold data. However, most of the existing federated learning algorithms for VP data are limited to synchronous computation. To improve the efficiency when the unbalanced computation/communication resources are common among the parties in the federated learning system, it is essential to develop asynchronous training algorithms for VP data while keeping the data privacy. In this article, we propose an asynchronous federated stochastic gradient descent (AFSGD-VP) algorithm and its two variance reduction variants, including stochastic variance reduced gradient (SVRG) and SAGA on the VP data. Moreover, we provide the convergence analyses of AFSGD-VP and its SVRG and SAGA variants under the condition of strong convexity and without any restrictions of staleness. We also discuss their model privacy, data privacy, computational complexities, and communication costs. To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for VP data with theoretical guarantees. Extensive experimental results on a variety of VP datasets not only verify the theoretical results of AFSGD-VP and its SVRG and SAGA variants but also show that our algorithms have much higher efficiency than the corresponding synchronous algorithms.
Augmentation Strategies for Learning with Noisy Labels

Imperfect labels are ubiquitous in real-world datasets. Several recent successful methods for training deep neural networks (DNNs) robust to label noise have used two primary techniques: filtering samples based on loss during a warm-up phase to curate an initial set of cleanly labeled samples, and using the output of a network as a pseudo-label for subsequent loss calculations. In this paper, we evaluate different augmentation strategies for algorithms tackling the "learning with noisy labels" problem. We propose and examine multiple augmentation strategies and evaluate them using synthetic datasets based on CIFAR-10 and CIFAR-100, as well as on the real-world dataset Clothing1M. Due to several commonalities in these algorithms, we find that using one set of augmentations for loss modeling tasks and another set for learning is the most effective, improving results on the state-of-the-art and other previous methods. Furthermore, we find that applying augmentation during the warm-up period can negatively impact the loss convergence behavior of correctly versus incorrectly labeled samples. We introduce this augmentation strategy to the state-of-the-art technique and demonstrate that we can improve performance across all evaluated noise levels. In particular, we improve accuracy on the CIFAR-10 benchmark at 90% symmetric noise by more than 15% in absolute accuracy, and we also improve performance on the Clothing1M dataset.
When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework

To minimize the effects of age variation in face recognition, previous work either extracts identity-related discriminative features by minimizing the correlation between identity- and age-related features, called age-invariant face recognition (AIFR), or removes age variation by transforming the faces of different age groups into the same age group, called face age synthesis (FAS); however, the former lacks visual results for model interpretation while the latter suffers from artifacts compromising downstream recognition. Therefore, this paper proposes a unified, multi-task framework to jointly handle these two tasks, termed MTL-Face, which can learn age-invariant identity-related representation while achieving pleasing face synthesis. Specifically, we first decompose the mixed face features into two uncorrelated components—identity- and age-related features—through an attention mechanism, and then decorrelate these two components using multi-task training and continuous domain adaption. In contrast to the conventional one-hot encoding that achieves group-level FAS, we propose a novel identity conditional module to achieve identity-level FAS, with a weight-sharing strategy to improve the age smoothness of synthesized faces. In addition, we collect and release a large cross-age face dataset with age and gender annotations to advance AIFR and FAS. Extensive experiments on five benchmark cross-age datasets demonstrate the superior performance of our proposed MTLFace over state-of-the-art methods for AIFR and FAS. We further validate MTLFace on two popular general face recognition datasets, showing competitive performance for face recognition in the wild. The source code and dataset are available at https://github.com/Hzzone/MTLFace.
Urdu Sentiment Analysis With Deep Learning Methods

Although over 169 million people in the world are familiar with the Urdu language and a large quantity of Urdu data is being generated on different social websites daily, very few research studies and efforts have been completed to build language resources for the Urdu language and examine user sentiments. The primary objective of this study is twofold: (1) develop a benchmark dataset for resource-deprived Urdu language for sentiment analysis and (2) evaluate various machine and deep learning algorithms for sentiment. To find the best technique, we compare two modes of text representation: count-based, where the text is represented using word n-gram feature vectors and the second one is based on fastText pre-trained word embeddings for Urdu. We consider a set of machine learning classifiers (RF, NB, SVM, AdaBoost, MLP, LR) and deep leaning classifiers (1D-CNN and LSTM) to run the experiments for all the feature types. Our study shows that the combination of word n-gram features with LR outperformed other classifiers for sentiment analysis task, obtaining the highest F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> score of 82.05% using combination of features.
A survey of color image watermarking: State-of-the-art and research directions

Color image watermarking has achieved great success in various applications such as copy protection, content authentication, tamper detection and recovery, etc. The present paper provides a detailed overview of watermarking along with its characteristics, evaluation metrics, and current applications. We then provide a comprehensive survey of different approaches along with their merits and limitations. Furthermore, we summarize each of the approaches in detail, including objectives, goals, embedding location, evaluation metrics, and weaknesses, and discuss recent challenges and their possible solutions. We believe this paper will provide readers with comprehensive insight into the field of watermarking research.