Reviewerly/exHarmony huggingface dataset:
Design and development of a vision‐based system for detection, tracking and recognition of isolated dynamic bare hand gesticulated characters

Abstract Detection and tracking are the vital stages to form the gesture trajectory in gesture recognition. It becomes more challenging when the variations in illumination, pose, position, occlusion, scale, speed, blurring effect and complex environment are introduced. Additionally, the background feature domination effect affects the existing deep learning models. A semantic segmentation model is implemented in this work to detect the bare hand to overcome these challenges. A pre‐trained network VGG‐16 is utilized by training with the proposed NITS S‐Net database. Evaluation of the SegNet model is done on EgoHands, Oxford and OUHands databases. To track the bare hand, a SegNet‐based detection and tracking approach is proposed using Kalman filter and point‐tracker. This model achieves 97.01% accuracy (a relative improvement of ~8% from the baseline models) at 0.068 s per frame computational time on NITS hand gesture database VIIIB. The gesticulated characters, that is, alphabets, numbers, operators, special characters, are gesticulated without any constraints on the pattern/strokes. To recognize these 95 multi‐stroke gestures, a deep convolutional neural network (DCNN) is presented using AlexNet. The DCNN model achieves 97.60% (a relative improvement of ~14% from the baseline models) accuracy on the NITS hand gesture database VIIIB merged. Evaluation of the handwritten EMNIST merged (balanced) database resulted in average recognition accuracy of 91.60%.
TIIA: A blockchain-enabled Threat Intelligence Integrity Audit scheme for IIoT

With the widespread deployment of the Industrial Internet of Things technology (IIoT), the cyber security threats of IIoT systems are increasing. Threat intelligence sharing is an effective way to resist cyber security threats. However, there is a contradiction between information integrity and the requirement of building a complete attack chain in the threat intelligence sharing and utilization. Due to the limited capacity of block storage pattern, it can be found that threat intelligence is prone to fragmentation on blockchain. In this paper, a blockchain-enabled Threat Intelligence Integrity Audit (TIIA) scheme for IIoT is proposed. This scheme can audit the integrity of threat intelligence in ciphertext state to ensure the confidentiality protection requirements of threat intelligence on blockchain. A double chain structure is presented in the TIIA scheme. The storage-chain is used to store threat intelligence ciphertext, while the audit-chain is designed to achieve the integrity audit of the corresponding fragment on the storage-chain. In addition, Paillier homomorphic encryption and searchable encryption are used to realize the confidentiality and ciphertext retrieval of threat intelligence on the storage-chain. A redundant block deletion algorithm is designed to reduce the audit-chain load and improve audit efficiency. The performance analysis shows that the proposed scheme can effectively reduce the computational and communication costs, and has a high audit efficiency.
Harmonious Multi-branch Network for Person Re-identification with Harder Triplet Loss

Recently, advances in person re-identification (Re-ID) has benefitted from use of the popular multi-branch network. However, performing feature learning in a single branch with uniform partitioning is likely to separate meaningful local regions, and correlation among different branches is not well established. In this article, we propose a novel harmonious multi-branch network (HMBN) to relieve these intra-branch and inter-branch problems harmoniously. HMBN is a multi-branch network with various stripes on different branches to learn coarse-to-fine pedestrian information. We first replace the uniform partition with a horizontal overlapped partition to cover meaningful local regions between adjacent stripes in a single branch. We then incorporate a novel attention module to make all branches interact by modeling spatial contextual dependencies across branches. Finally, in order to train the HMBN more effectively, a harder triplet loss is introduced to optimize triplets in a harder manner. Extensive experiments are conducted on three benchmark datasets — DukeMTMC-reID, CUHK03, and Market-1501 — demonstrating the superiority of our proposed HMBN over state-of-the-art methods.
Foreign Body Detection in Rail Transit Based on a Multi-Mode Feature-Enhanced Convolutional Neural Network

Detection of railway traffic objects is an important task during train driving and is implemented to ensure safe driving. Although object detection has been investigated for years, many challenges exist in precisely detecting railway objects under complex railway scenes. These challenges mainly include adverse weather states, various railway backgrounds, diverse railway objects, and low-quality images. To address these issues, we introduce a novel deep learning method, called a multi-mode feature enhanced convolutional neural network (MMFE-Net), for accurate railway object detection. The network mainly consists of three modules. 1) An improved cross-stage partial connection darknet53 (CSPDarknet53), called adaptive dilated cspdarknet53, is used as our backbone to reduce image information loss. 2) A spatial feature extraction module is used to improve the feature extraction ability of the model for blurred objects and objects in a complicated background. 3) We introduce an attention fusion enhance module to strengthen the context information between adjacent feature maps to accurately detect multiscale and small objects. The proposed method achieves 0.9439 mAP and 79 FPS with an input size of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$640\times 640$ </tex-math></inline-formula> pixels on the railway traffic dataset, and its performance is better than that of YOLOv4. Moreover, it is feasible to apply MMFE-Net into practical applications of railway object detection.
MDOPE: Efficient multi-dimensional data order preserving encryption scheme

Nowadays, tremendous information technology industries resort to cloud servers to store data with an outsourcing approach to extend their storage and computation power. This, however, also leads to privacy and security issues of unprotected data against curious cloud servers. The most common solution currently is to encrypt the data before uploading it. The problem is that encrypted data is out of control and users also cannot transfer the searching job to cloud serves anymore. Specifically, the order preserving encryption (OPE) provides an efficient solution to the order of plaintexts. Existing OPE schemes focus on single-dimensional data, and fail to effectively process multi-dimensional data. In this paper, we propose a multi-dimensional data order preserving encryption scheme MDOPE allowing fine-grained multi-dimensional range queries. Our scheme constructs query indexes for each dimension of the data based on the order preserving encryption network. In particular, the proposed scheme ensures that no external entity, including the cloud server, can obtain additional information other than the order of ciphertexts during the whole query process.
Word Level Script Identification Using Convolutional Neural Network Enhancement for Scenic Images

Script identification from complex and colorful images is an integral part of the text recognition and classification system. Such images may contain twofold challenges: (1) Challenges related to the camera like blurring effect, non-uniform illumination and noisy background, and so on, and (2) Challenges related to the text shape, orientation, and text size. The present work in this area is much focused on non-Indian scripts. In contrast, Gurumukhi, Hindi, and English scripts play a vital role in communication among Indians and foreigners. In this article, we focus on the above said challenges in the field of identifying the script. Additionally, we have introduced a new dataset that contains Hindi, Gurumukhi, and English scripts from scenic images collected from different sources. We also proposed a CNN-based model, which is capable of distinguishing between the scripts with good accuracy. Performance of the method has been evaluated for own dataset, i.e., NITJDATASET and other benchmarked datasets available for Indian scripts, i.e., CVSI-2015 (Task-1 and Task 4) and ILST. This work is an extension to find the script from strict text background.
The variability and seasonality in the ratio of photosynthetically active radiation to solar radiation: A simple empirical model of the ratio

A constant ratio of photosynthetically active radiation (PAR) to solar radiation (SR) (about 0.45–0.46) has been used in many ecological studies to convert SR into PAR by multiplication. The constant ratio is useful and convenient. However, there is no general agreement on whether the ratio is 0.45 or 0.46 and how distributed globally. Accordingly, many local empirical ratios have been reported. This study aims to demonstrate a global distribution of the ratio and its changing range. Thus, we created two simple empirical models to estimate the ratio based on the in-situ climatic data to achieve the aim. The models were created based on accurate data of SR and PAR observed using a direct and diffuse separation method at Tateno in Tsukuba, Japan. At three validation sites in Japan, the ratio could be estimated with an error within 3%, a considerable reduction from 15% in using a constant. The numerical model also produced the ratio within approximately 3% errors. Using the proposed model, we demonstrated that the annual mean of the ratio had a range from 0.409 to 0.477. The results will contribute to the uncertainty estimation when using a constant ratio.
SOMA: Solving Optical Marker-Based MoCap Automatically

Marker-based optical motion capture (mocap) is the "gold standard" method for acquiring accurate 3D human motion in computer vision, medicine, and graphics. The raw output of these systems are noisy and incomplete 3D points or short tracklets of points. To be useful, one must associate these points with corresponding markers on the captured subject; i.e. "labelling". Given these labels, one can then "solve" for the 3D skeleton or body surface mesh. Commercial auto-labeling tools require a specific calibration procedure at capture time, which is not possible for archival data. Here we train a novel neural network called SOMA, which takes raw mocap point clouds with varying numbers of points, labels them at scale without any calibration data, independent of the capture technology, and requiring only minimal human intervention. Our key insight is that, while labeling point clouds is highly ambiguous, the 3D body provides strong constraints on the solution that can be exploited by a learning-based method. To enable learning, we generate massive training sets of simulated noisy and ground truth mocap markers animated by 3D bodies from AMASS. SOMA exploits an architecture with stacked self-attention elements to learn the spatial structure of the 3D body and an optimal transport layer to constrain the assignment (labeling) problem while rejecting outliers. We extensively evaluate SOMA both quantitatively and qualitatively. SOMA is more accurate and robust than existing state of the art research methods and can be applied where commercial systems cannot. We automatically label over 8 hours of archival mocap data across 4 different datasets captured using various technologies and output SMPL-X body models. The model and data is released for research purposes at https://soma.is.tue.mpg.de/.
Blockchain-Based Audio Watermarking Technique for Multimedia Copyright Protection in Distribution Networks

Copyright protection in multimedia protection distribution is a challenging problem. To protect multimedia data, many watermarking methods have been proposed in the literature. However, most of them cannot be used effectively in a multimedia distribution network (MDN) as they are not designed to support multi-layer watermark embedding. Multi-layer watermarking mechanisms were developed to protect multimedia data across different layers in an MDN. However, in those mechanisms, we need to trust the entities in the MDN, such as regional and country distributors. To overcome this potential drawback, in this article, we propose a novel privacy protection mechanism for MDNs by combining the advantages of both blockchain and watermarking technologies. A specifically designed watermarking algorithm is used to link the copyright information with the audio file, while a novel blockchain-based smart contract mechanism is developed to enforce the proper functioning of each entity in the distribution network. Moreover, the new audio mechanism is computationally efficient. Although audio signals are used to show the effectiveness of the proposed mechanism, the proposed approach can easily be extended to other multimedia objects, such as an image. The validity of the proposed mechanism is demonstrated by our simulation results. The proposed mechanism can benefit multimedia production companies and other entities in the MDN.
MIST-Tacotron: End-to-End Emotional Speech Synthesis Using Mel-Spectrogram Image Style Transfer

With the development of voice synthesis technology using deep learning, voice synthesis research that expresses the characteristics and emotions of speakers is actively being conducted. Current technology does not satisfactorily express various emotions and characteristics for speakers with very low or high vocal ranges and for speakers with dialects. In this paper, we propose mel-spectrogram image transfer (MIST)-Tacotron, a Tacotron 2-based speech synthesis model that adds a reference encoder with an image style transfer module. The proposed method is a technique that adds image style transfer to the existing Tacotron 2 model and extracts the speaker&#x2019;s feature from the reference mel-spectrogram using a pre-trained deep learning model. Through the extracted feature, the style such as pitch, tone, and duration of the speaker are trained to express the style and emotion of the speaker more clearly. To extract the speaker&#x2019;s style independently from the speaker&#x2019;s timbre and emotion, the ID value for the speaker and the ID value for the emotional state were used as inputs. Performance is evaluated by F0 voiced error (FVE), F0 gross pitch error (F0 GPE), mel-cepstral distortion (MCD), band aperiodicity distortion (BAPD), voiced/unvoiced error (VUVE), false positive rate (FPR), and false negative rate (FNR). The performance of the proposed model was observed to have lower error values than the existing models, GST (Global Style Token) Tacotron and VAE (Variational Autoencoder) Tacotron. As a result of measuring mean opinion score (MOS), the sound quality of the proposed model received the highest score in terms of emotional expression and speaker style reflection.
FlauBERT vs. CamemBERT: Understanding patient's answers by a French medical chatbot

In a number of circumstances, obtaining health-related information from a patient is time-consuming, whereas a chatbot interacting efficiently with that patient might help saving health care professional time and better assisting the patient. Making a chatbot understand patients' answers uses Natural Language Understanding (NLU) technology that relies on 'intent' and 'slot' predictions. Over the last few years, language models (such as BERT) pre-trained on huge amounts of data achieved state-of-the-art intent and slot predictions by connecting a neural network architecture (e.g., linear, recurrent, long short-term memory, or bidirectional long short-term memory) and fine-tuning all language model and neural network parameters end-to-end. Currently, two language models are specialized in French language: FlauBERT and CamemBERT. This study was designed to find out which combination of language model and neural network architecture was the best for intent and slot prediction by a chatbot from a French corpus of clinical cases. The comparisons showed that FlauBERT performed better than CamemBERT whatever the network architecture used and that complex architectures did not significantly improve performance vs. simple ones whatever the language model. Thus, in the medical field, the results support recommending FlauBERT with a simple linear network architecture.
Modifying quantum optical states by zero-photon subtraction

The process of single-photon subtraction (SPS) is known to dramatically alter the properties of certain quantum optical states. Somewhat surprisingly, subtracting zero photons can also modify quantum states and has practical applications in quantum communication. Here we experimentally investigate zero-photon subtraction (ZPS) using a wide variety of input states and conditional measurements based on actively detecting zero photons in one output port of a variable beamsplitter. We find that SPS and ZPS can exhibit complementary behavior depending on the photon statistics of the input states, and highlight deeper connections with Mandel's $Q$-parameter for classifying quantum states.
Empowering Next-Generation IoT WLANs Through Blockchain and 802.11ax Technologies

Blockchain emerges as a potential solution for enabling distributed management and accountability of wireless Internet of Thing (IoT) networks. In addition, blockchain offers increased IoT security, as it involves every single node around the network to verify and approve new transactions. On the other hand, IEEE 802.11ax, a recently concluded WiFi standard aimed at achieving high network efficiency, plays a key role for solving spectrum sharing and cross-technology coexistence among WiFi and 6G cellular users, one of the key challenges faced by next-generation wireless systems. In this paper, we leverage the power of blockchain and 802.11ax technologies to propose a medium access control (MAC) protocol design for future IoT wireless local area networks (WLANs), also referred to as wireless IoT-Blockchain networks setup. Our simulation results show that the proposed protocol design enhances transmission latency and achievable network throughput.
High-quality domain expert finding method in CQA based on multi-granularity semantic analysis and interest drift

Expert finding is an important research field in community question answering (CQA). Traditional expert finding methods mainly exploit topic analysis and authority calculation methods to identify high-quality experts in certain fields. To avoid recommending questions to those experts who do not display the willingness or ability to provide high-quality answers, user interest drift and user quality should be considered. This study proposes a novel method named high-quality domain expert finding in CQA based on multi-granularity semantic analysis and interest drift (HQExpert). Firstly, HQExpert considers different semantic granularities by employing two models, a coarse-grained topic model LC-LDA and a fine-grained model (BERT), to capture the domain information of questions and users more accurately. Secondly, to address the diverse interests of the users, a user interest drift model in HQExpert is developed to dynamically represent the changes in the interests of the users at different periods. In addition, a user quality model is developed to further optimize the professional level of the user, finding experts who can provide high-quality answers and are interested in the current question. Finally, extensive experiments on two datasets from different domains demonstrate that the proposed HQExpert model can significantly improve the accuracy of finding high-quality experts.
Evolving multi-user fuzzy classifier systems integrating human uncertainty and expert knowledge

Evolving classifiers and especially evolving fuzzy classifiers have been established as a prominent technique for addressing the recent demands in building classifiers in an incremental open-loop manner, e.g. for the purpose of processing data streams online. So far, the focus lies on classifiers which are obtained based on the input and/or feedback in the form of target labels provided by a single user/expert. In this paper, we propose three variants of evolving multi-user fuzzy classifier systems (EFCS-MU), where multiple users may provide their label feedback: i) ensembled single-user classifiers system, which allows a separate classifier training per user and embeds an advanced aggregation strategy (→ ensembling on a model level), ii) consensus all-user classifier system, where a joint classifier is established for all users based on consensus labelings (→ ensembling on a label level), iii) shift-work all-user classifier system, where a joint classifier is established for all users based on the classical shift-work concept. The classifiers are incrementally evolved by a single-pass learning approach embedding the autonomous evolution of new rules on demand; it integrates an unsupervised evolving clustering technique for rule partitioning, thus the same partition is established in all single-user classifiers, only the consequents in the form of class confidence vectors typically differ among the users due to their different labelings. This offers direct explainability of the varying users' annotation behaviors. The possible different experience levels of the users in relation to the process behind and possible ambiguities among the provided users' labels are handled by the proper integration of uncertainty levels into the update of the classifier(s). Furthermore, a concept is presented as to how to adequately integrate possibly available expert rules for a particular newly (on-the-fly) arising class (or in advance for several classes). Finally, an on-line active learning (oAL) strategy is demonstrated, to select only the most important samples to be labelled and thus reduce users' labeling costs, ensuring economically practicable applicability. The approach was successfully evaluated on two real-world application scenarios, one stemming for a visual inspection scenario, where four users check the quality of the imprint of compact discs and are affected by different experience levels, and one from a nursery school employment ranking application, where a new class was introduced later. The results provide insights into the performance behavior of the three different multi-user classifier variants under different circumstances (with and without expert rules, uncertainty integration, different labelling budgets etc.), including comparisons based on on-line accuracy trends versus the economy of the labeling effort.
KWickChat: A Multi-Turn Dialogue System for AAC Using Context-Aware Sentence Generation by Bag-of-Keywords

We present KWickChat (Keyword Quick Chat): a multi-turn augmentative and alternative communication (AAC) dialogue system for nonspeaking individuals with motor disabilities. The central objective of KWickChat is to reduce the communication gap between nonspeaking and speaking partners by exploring a sentence-based text entry system that automatically generates suitable sentences for the nonspeaking partner based on keyword entry. The system is underpinned by a GPT-2 language model and leverages context information, including dialogue history and persona tags, to improve the quality of the generated responses. We evaluate the system by analyzing the functional design and decomposing it into key functions and parameters that are systematically investigated using envelope analysis. We pursue this methodology as a necessary precursor to evaluation with AAC users. Our results show that with word prediction and with a threshold word error rate of 0.65, the keystroke savings of the KWickChat system is around 71%. To complement the envelope analysis, we also recruited two human judges to evaluate the semantic consistency between 400 sentences generated by KWickChat and reference sentences. Both judges reported a median rating of 4 on a scale from 1 (very bad) to 5 (very good) for the best generated sentence in each exchange and achieved an inter-rater reliability of 0.92 across all 400 sentences judged.