Reviewerly/exHarmony huggingface dataset:
Transformers Meet Small Datasets

The research and application areas of transformers have been extensively enlarged due to the success of vision transformers (ViTs). However, due to the lack of local content acquisition capabilities, the pure transformer architectures cannot be trained directly on small datasets. In this work, we first propose a new hybrid model by combining the transformer and convolution neural network (CNN). The proposed model improves the classification ability on small datasets. This is accomplished by introducing more convolution operations in the transformer’s two core sections: 1) Instead of the original multi-head attention mechanism, we design a convolutional parameter sharing multi-head attention (CPSA) block that incorporates the convolutional parameter sharing projection in the attention mechanism; 2) the feed-forward network in each transformer encoder block is replaced with a local feed-forward network (LFFN) block that introduces a sandglass block with more depth-wise convolutions to provide more locality to the transformers. We achieve state-of-the-art results when training from scratch on 4 small datasets as compared with the transformers and CNNs without extensive computing resources and auxiliary training. The proposed strategy opens up new paths for the application of transformers on small datasets.
3D Gesture Recognition and Adaptation for Human–Robot Interaction

Gesture-based human-robot interaction has been an important area of research in recent years. The primary aspect for the researchers has always been to create a gesture detection system that is insensitive to lighting and backdrop surroundings. This research proposes a 3D gesture recognition and adaption system based on Kinect for human-robot interaction. The framework consists of four modules, i.e., pointing gesture recognition, 3D dynamic gesture recognition, gesture adaptation, and robot navigation. The proposed dynamic gesture recognition module employs three distinct classifiers: HMM, Multi-class SVM, and CNN. The adaptation module can adapt to new and unrecognized gestures applying semi-supervised self-adaptation or user consent-based adaptation. A Graphical User Interface (GUI) is built for training and testing the proposed system on the fly. A simple simulator along with two different robot-navigation algorithms are developed to test robot navigation based on the recognized gestures. The framework is trained and tested through a five-fold cross-validation method with a total of 3,600 gesture instances of ten predefined gestures performed by 24 persons (Three age categories: Young, Middle Aged, Adult; each with 1,200 gestures). The proposed system achieves a maximum accuracy score of 95.67% with HMM for Middle Aged category, 92.59% with SVM for Middle Aged category, and 89.58% with CNN for Young category in dynamic gesture recognition. Considering all the three age categories, the system achieves an average accuracy of 94.61%, 91.95%, and 88.97% in recognizing dynamic gestures with HMM, SVM, and CNN respectively. Moreover, the system recognizes pointing gestures in real-time.
Efficient Driver Drunk Detection by Sensors: A Manifold Learning-Based Anomaly Detector

This study presents an effective data-driven anomaly detection scheme for drunk driving detection.Specifically, the proposed anomaly detection approach amalgamates the desirable features of the t-distributed stochastic neighbor embedding (t-SNE) as a feature extractor with the Isolation Forest (iF) scheme to detect drivers' drunkenness status.We used the t-SNE model to exploit its capacity in reducing the dimensionality of nonlinear data by preserving the local and global structures of the input data in the feature space to obtain good detection.At the same time, the iF scheme is an effective and unsupervised treebased approach to achieving good detection of anomalies in multivariate data.This approach only employs normal events data to train the detection model, making them more attractive for detecting drunk drivers in practice.To verify the detection capacity of the proposed t-SNE-iF approach in reliably detecting drivers with excess alcohol, we used publically available data collected using a gas sensor, temperature sensor, and a digital camera.The overall detection system proved a high detection performance with AUC around 95%, demonstrating the proposed approach's robustness and reliability.Furthermore, compared to the Principal Component Analysis (PCA), Incremental PCA (IPCA), Independent component analysis (ICA), Kernel PCA (kPCA), and Multi-dimensional scaling (MDS)-based iForest, EE, and LOF detection schemes, the proposed t-SNE-based iF scheme offers superior detection performance of drunk driver status.
Analysis of the Consensus Protocols used in Blockchain Networks – An overview

Distributed ledger technologies have opened up a wide variety of possibilities in various application sectors like Blockchain, Hashgraph, Holochain, Directed Acyclic Graph (DAG) and Radix. The consensus mechanism eliminates a centralized controller from altering the “content” or how it should be recorded on a blockchain. The consensus protocol is the defining technology behind the performance of blockchain, among other important components. Many people believe that data stored in a Blockchain is and will always be safe because of its decentralization and encryption. Consensus layer aids in measuring performance of the blockchain. Consensus procedures are essential for the long-term stability of a Blockchain system. Blockchain technology is built on consensus. Consensus protocols are invented to enhance the efficiency of Blockchain while also satisfying the specific needs of different application domains. Consensus protocol increases the efficiency of the blockchain by increasing the throughput of the system. PoW, PoS, DPoS, PoET, PoA and PBFT are the protocols reviewed in this paper. In this paper, we analyze and compare the strengths and weaknesses of consensus protocols. We have considered each consensus protocol’s performance metrics like throughput, scalability, decentralization accessibility and applications for blockchain networks. This survey will offer blockchain developers and researchers a thorough perspective of current consensus protocols and will aid in the development of future protocols.
Adaptive Face Forgery Detection in Cross Domain

It is necessary to develop effective face forgery detection methods with constantly evolving technologies in synthesizing realistic faces which raises serious risks on malicious face tampering. A large and growing body of literature has investigated deep learning-based approaches, especially those taking frequency clues into consideration, have achieved remarkable progress on detecting fake faces. The method based on frequency clues result in the inconsistency across frames and make the final detection result unstable even in the same deepfake video. So, these patterns are still inadequate and unstable. In addition to this, the inconsistency problem in the previous methods is significantly exacerbated due to the diversities among various forgery methods. To address this problem, we propose a novel deep learning framework for face forgery detection in cross domain. The proposed framework explores on mining the potential consistency through the correlated representations across multiple frames as well as the complementary clues from both RGB and frequency domains. We also introduce an instance discrimination module to determine the discriminative results center for each frame across the video, which is a strategy that adaptive adjust with during inference.
Detecting and Recovering Sequential DeepFake Manipulation

Since photorealistic faces can be readily generated by facial manipulation technologies nowadays, potential malicious abuse of these technologies has drawn great concerns. Numerous deepfake detection methods are thus proposed. However, existing methods only focus on detecting one-step facial manipulation. As the emergence of easy-accessible facial editing applications, people can easily manipulate facial components using multi-step operations in a sequential manner. This new threat requires us to detect a sequence of facial manipulations, which is vital for both detecting deepfake media and recovering original faces afterwards. Motivated by this observation, we emphasize the need and propose a novel research problem called Detecting Sequential DeepFake Manipulation (Seq-DeepFake). Unlike the existing deepfake detection task only demanding a binary label prediction, detecting Seq-DeepFake manipulation requires correctly predicting a sequential vector of facial manipulation operations. To support a large-scale investigation, we construct the first Seq-DeepFake dataset, where face images are manipulated sequentially with corresponding annotations of sequential facial manipulation vectors. Based on this new dataset, we cast detecting Seq-DeepFake manipulation as a specific image-to-sequence (e.g. image captioning) task and propose a concise yet effective Seq-DeepFake Transformer (SeqFakeFormer). Moreover, we build a comprehensive benchmark and set up rigorous evaluation protocols and metrics for this new research problem. Extensive experiments demonstrate the effectiveness of SeqFakeFormer. Several valuable observations are also revealed to facilitate future research in broader deepfake detection problems.
Aesthetic Text Logo Synthesis via Content-aware Layout Inferring

Text logo design heavily relies on the creativity and expertise of professional designers, in which arranging element layouts is one of the most important procedures. However, few attention has been paid to this task which needs to take many factors (e.g., fonts, linguistics, topics, etc.) into consideration. In this paper, we propose a content-aware layout generation network which takes glyph images and their corresponding text as input and synthesizes aesthetic layouts for them automatically. Specifically, we develop a dual-discriminator module, including a sequence discriminator and an image discriminator, to evaluate both the character placing trajectories and rendered shapes of synthesized text logos, respectively. Furthermore, we fuse the information of linguistics from texts and visual semantics from glyphs to guide layout prediction, which both play important roles in professional layout design. To train and evaluate our approach, we construct a dataset named as TextLogo3K, consisting of about 3,500 text logo images and their pixel-level annotations. Experimental studies on this dataset demonstrate the effectiveness of our approach for synthesizing visually-pleasing text logos and verify its superiority against the state of the art.
Digital Banking

Due to the exponential growth of the internet, smartphones, and communication technologies during the last two decades, the digital banking sector has enormously advanced in terms of user-friendly, efficient, and fast financial transactions. Digital banking also plays a significant role as an enabler of cashless transactions in the economic crisis caused by the COVID-19 pandemic. The study investigates the challenges, technology, and future research agenda of digital banking. The paper follows the manifestation of Kitchenham's SLR protocol. Six databases were used to determine articles that match the criteria. The study considers recent articles, which have been published from 2015 to 2021. Sixty-seven papers have been selected, extracted, and analyzed. The result highlights issues related to technology, organization, people, process, environment, customers, security, and risk, which become challenges in digital banking innovation. This research presents suggestions for future research directions, which will be beneficial to practitioners and scholars around the globe.
Asymmetric Cross-Scale Alignment for Text-Based Person Search

Text-based person search (TBPS) is of significant importance in intelligent surveillance, which aims to retrieve pedestrian images with high semantic relevance to a given text description. This retrieval task is characterized with both modal heterogeneity and fine-grained matching. To implement this task, one needs to extract multi-scale features from both image and text domains, and then perform the cross-modal alignment. However, most existing approaches only consider the alignment confined at their individual scales, e.g., an image-sentence or a region-phrase scale. Such a strategy adopts the presumable alignment in feature extraction, while overlooking the cross-scale alignment, e.g., image-phrase. In this paper, we present a transformer-based model to extract multi-scale representations, and perform Asymmetric Cross-Scale Alignment (ACSA) to precisely align the two modalities. Specifically, ACSA consists of a global-level alignment module and an asymmetric cross-attention module, where the former aligns an image and texts on a global scale, and the latter applies the cross-attention mechanism to dynamically align the cross-modal entities in region/image-phrase scales. Extensive experiments on two benchmark datasets CUHK-PEDES and RSTPReid demonstrate the effectiveness of our approach. Codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mul-hjh/ACSA</uri> .
MVSalNet: Multi-view Augmentation for RGB-D Salient Object Detection

RGB-D salient object detection (SOD) enjoys significant advantages in understanding 3D geometry of the scene. However, the geometry information conveyed by depth maps are mostly under-explored in existing RGB-D SOD methods. In this paper, we propose a new framework to address this issue. We augment the input image with multiple different views rendered using the depth maps, and cast the conventional single-view RGB-D SOD into a multi-view setting. Since different views captures complementary context of the 3D scene, the accuracy can be significantly improved through multi-view aggregation. We further design a multi-view saliency detection network (MVSalNet), which firstly performs saliency prediction for each view separately and incorporates multi-view outputs through a fusion model to produce final saliency prediction. A dynamic filtering module is also designed to facilitate more effective and flexible feature extraction. Extensive experiments on 6 widely used datasets demonstrate that our approach compares favorably against state-of-the-art approaches.
Digital Fashion Revolutions: Supply Chain Transparency, Digitalization and the Non-Disclosure Paradox

The emergence of digital technology as a ‘cure-all’ for sustainable practice has captured the imagination of observers and entrepreneurs alike. Among these technologies, blockchain has been cited as the ideal tool to optimize supply chain transparency. However, despite the abundance of effusively disseminated information in the media, the lack of blockchain applications that are universally accessible, and their negligible uptake raises doubts as to its utility. In particular, small to medium enterprises (SMEs) have for several reasons been slow to adopt blockchain technologies. ‘Digital hesitancy’; lack of common data standards; complex and tedious data collection and transfer; immaturity of the technology; no effective universal platform; lack of resources; and reluctance to share data with perceived competitors are common obstacles. This study explores one of these barriers to adoption, that is, the reluctance to disclose supply chain information to potential competitors. Taking a qualitative approach, the study analyses the current perception of blockchain enabled supply chain transparency through interviews with small scale fashion firms and technology start-ups. Applying a lens of technology adoption theories, the study seeks to understand how supply chain transparency might be satisfactorily managed and even accelerated through technology uptake. A misunderstanding of the software’s capabilities is evident. Many firms seem unaware that the software can be applied to provide managed access to information—arguably providing an advantage over extant transparency measures such as public self disclosure or reliance on third party certifications. This study identifies that because of lack of ‘education’, firms are ill-informed and under-utilizing technologies that are potentially more advantageous than current analogue approaches—that could circumvent the information disclosure paradox.
Agile Scrum Applied in Agricultural Processes

Abstract This paper presents a case study of interdisciplinary subjects, more precisely the AGILE SCRUM specific to software development applied in agricultural processes to increase the productivity, maximize the profit and to emphasize the benefits of AGILE methodology which is suitable in every field of industry. Starting with theoretical aspects about Agile project management approach and philosophy, including values and principles, continuing with SCRUM framework, is emphasized the applicability of the method in various fields of industry, including agriculture.
DiSparse: Disentangled Sparsification for Multitask Model Compression

Despite the popularity of Model Compression and Mul-titask Learning, how to effectively compress a multitask model has been less thoroughly analyzed due to the chal-lenging entanglement of tasks in the parameter space. In this paper, we propose DiSparse, a simple, effective, and first-of-its-kind multitask pruning and sparse training scheme. We consider each task independently by disentangling the importance measurement and take the unani-mous decisions among all tasks when performing parame-ter pruning and selection. Our experimental results demon-strate superior performance on various configurations and settings compared to popular sparse training and pruning methods. Besides the effectiveness in compression, DiS-parse also provides a powerful tool to the multitask learning community. Surprisingly, we even observed better per-formance than some dedicated multitask learning methods in several cases despite the high model sparsity enforced by DiSparse. We analyzed the pruning masks generated with DiSparse and observed strikingly similar sparse net-work architecture identified by each task even before the training starts. We also observe the existence of a “water-shed” layer where the task relatedness sharply drops, implying no benefits in continued parameters sharing. Our code and models will be available at: https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression.
Learning to Answer Questions in Dynamic Audio-Visual Scenarios

In this paper, we focus on the Audio-Visual Question Answering (AVQA) task, which aims to answer questions regarding different visual objects, sounds, and their associations in videos. The problem requires comprehensive multimodal understanding and spatio-temporal reasoning over audio-visual scenes. To benchmark this task and facilitate our study, we introduce a large-scale MUSIC-AVQA dataset, which contains more than 45K question-answer pairs covering 33 different question templates spanning over different modalities and question types. We develop several baselines and introduce a spatio-temporal grounded audio-visual network for the AVQA problem. Our results demonstrate that AVQA benefits from multisensory perception and our model outperforms recent A-, V-, and AVQA approaches. We believe that our built dataset has the potential to serve as testbed for evaluating and promoting progress in audio-visual scene understanding and spatio-temporal reasoning. Code and dataset: http://gewu-lab.github.io/MUSIC-AVQA/
Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model

Amodal completion is a visual task that humans perform easily but which is difficult for computer vision algorithms. The aim is to segment those object boundaries which are occluded and hence invisible. This task is particularly challenging for deep neural networks because data is difficult to obtain and annotate. Therefore, we formulate amodal segmentation as an out-of-task and out-of-distribution generalization problem. Specifically, we replace the fully connected classifier in neural networks with a Bayesian generative model of the neural network features. The model is trained from non-occluded images using bounding box annotations and class labels only, but is applied to generalize out-of-task to object segmentation and to generalize out-of-distribution to segment occluded objects. We demonstrate how such Bayesian models can naturally generalize beyond the training task labels when they learn a prior that models the object's background context and shape. Moreover, by leveraging an outlier process, Bayesian models can further generalize out-of-distribution to segment partially occluded objects and to predict their amodal object boundaries. Our algorithm outperforms alternative methods that use the same supervision by a large margin, and even outperforms methods where annotated amodal segmentations are used during training, when the amount of occlusion is large. Code is publicly available at https://github.com/YihongSun/Bayesian-Amodal.
Contrastive Positive Mining for Unsupervised 3D Action Representation Learning

Recent contrastive based 3D action representation learning has made great progress. However, the strict positive/negative constraint is yet to be relaxed and the use of non-self positive is yet to be explored. In this paper, a Contrastive Positive Mining (CPM) framework is proposed for unsupervised skeleton 3D action representation learning. The CPM identifies non-self positives in a contextual queue to boost learning. Specifically, the siamese encoders are adopted and trained to match the similarity distributions of the augmented instances in reference to all instances in the contextual queue. By identifying the non-self positive instances in the queue, a positive-enhanced learning strategy is proposed to leverage the knowledge of mined positives to boost the robustness of the learned latent space against intra-class and inter-class diversity. Experimental results have shown that the proposed CPM is effective and outperforms the existing state-of-the-art unsupervised methods on the challenging NTU and PKU-MMD datasets.
Exploring E-Commerce Product Experience Based on Fusion Sentiment Analysis Method

With the speedy development of e-commerce, a growing number of customers tend to share their subjective perceptions of the product or service on the Internet.This phenomenon makes the commercial value of online reviews increasingly prominent.In this context, how to gain insights into consumers' perceptions and attitudes from massive comments has become a hot-button topic.Addressing this requirement, this paper developed a fusion sentiment analysis method combining textual analysis techniques with machine learning algorithms, aiming to mine online product experience.The method mainly consists of three steps.Firstly, inspired by the sensitivity of sentiment dictionary to emotional information, we utilize the dictionary to extract sentiment features.Afterward, the SVM algorithm is adopted to identify sentiment polarities of reviews.Based on this, sentiment topics are extracted from reviews through the LDA model.Furthermore, to avoid the omission of emotional information, the dictionary is extended based on semantic similarity.Meanwhile, in this research, the fact that words in reviews have unequal sentiment contribution, which has been neglected in existing studies, is taken into account.Specifically, we introduce the weighting method to measure the sentiment contribution.Finally, the investigation of consumers' reading experiences of online books on Amazon has verified the feasibility and validity of the method.The results demonstrate that the method accurately determines reviews' emotional tendencies and captures elements affecting reading experiences from reviews.Overall, the research provides an effective way to mine online product experience and track customers' demands, thereby strongly supporting future product improvement and marketing strategy optimization.