Reviewerly/exHarmony huggingface dataset:
Using YOLOv5 Algorithm to Detect and Recognize American Sign Language

Sign language is a system of communication using visual gestures and signs, and is generally used by people who have speech or hearing difficulties. To make those people included in the verbal communication community it's important to under-stand the gesture they're using to communicate. Often people who do not use the gesture in real life could not understand what the gesture represents. In this paper, we're proposing a solution to detect the alphabet and numbers that are each gesture providing. There are already some methods proposed that use deep learning for recognizing sign language. However, the effective uses of these models are limited. We are proposing a YOLOv5 based solution as it's lightweight, fast, and has good accuracy. We use a benchmark data set (MU_HandImages_ASL) to train and evaluate the model. We achieved 95% precision, 97% recall, 98% map@0.5, and 98% map@0.5:0.95 score which is adequate to recognize the gesture in real-time.
Framework of Blockchain-Supported E-Commerce Platform for Small and Medium Enterprises

Small and medium enterprises (SMEs) play an important role in promoting the national economy and providing employment opportunities. E-commerce platforms may facilitate transactions between these enterprises. However, transactions on the e-commerce platform are non-face-to-face, strongly virtual, and variable, resulting in SMEs often facing moral hazards and opportunistic behaviors. Blockchain is a distributed ledger consisting of an encryption algorithm, consensus mechanism, and smart contracts, having the characteristics of authenticity, security, transparency, and unforgeability. Thus, the problems faced by SMEs can be resolved by combining e-commerce platforms and blockchain technology. In this study, we first provide a conceptual framework for blockchain-supported e-commerce platforms for SMEs. Second, based on this conceptual framework, we build a total business architecture. Finally, we propose three key applications to illustrate how the platform facilitates SMEs in solving financing and trading problems. This study provides significant guidance for the operation and management of blockchain-supported e-commerce platforms for these enterprises.
Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing

BERT is the most recent Transformer-based model that achieves state-of-the-art performance in various NLP tasks. In this paper, we investigate the hardware acceleration of BERT on FPGA for edge computing. To tackle the issue of huge computational complexity and memory footprint, we propose to fully quantize the BERT (FQ-BERT), including weights, activations, softmax, layer normalization, and all the intermediate results. Experiments demonstrate that the FQ-BERT can achieve 7.94× compression for weights with negligible performance loss. We then propose an accelerator tailored for the FQ-BERT and evaluate on Xilinx ZCU102 and ZCU11 FPGA. It can achieve a performance-per-watt of 3.18 fps/W, which is 28.91× and 12.72× over Intel(R) Core(TM) i7-8700 CPU and NVIDIA K80 GPU, respectively.
Impact of the learners diversity and combination method on the generation of heterogeneous classifier ensembles

Ensembles of classifiers is a proven approach in machine learning with a wide variety of research works. The main issue in ensembles of classifiers is not only the selection of the base classifiers, but also the combination of their outputs. According to the literature, it has been established that much is to be gained from combining classifiers if those classifiers are accurate and diverse. However, it is still an open issue how to define the relation between accuracy and diversity in order to define the best possible ensemble of classifiers. In this paper, we propose a novel approach to evaluate the impact of the diversity of the learners on the generation of heterogeneous ensembles. We present an exhaustive study of this approach using 27 different multiclass datasets and analysing their results in detail. In addition, to determine the performance of the different results, the presence of labelling noise is also considered.
Probabilistic post-processing of gridded atmospheric variables and its application to site adaptation of shortwave solar radiation

Site adaptation refers to procedures for correcting systematic errors in an extended period of gridded modeled data using a short period of ground-based measurements used as unbiased reference. Traditionally, site adaptation leverages a single gridded product and issues point predictions. Currently, remote-sensed and reanalysis data are available from different sources providing multiple versions of estimates of a same atmospheric variable, for any location on Earth. These datasets allow what is called an ensemble prediction. In this regard, this contribution proposes a probabilistic site-adaption framework, and describes how one can use parametric and nonparametric techniques within the framework. On top of the stand-alone probabilistic site-adaption methods, heuristics are optionally used to combine quantiles , to further improve the accuracy of site adaptation. To exemplify the framework, global horizontal irradiance data from 26 sites worldwide with different climate characteristics and weather regimes are used to side-adapt the corresponding predictions from up to 5 satellite-derived databases and 2 reanalyses spanning various periods, collectively. It is found that the proposed site-adaptation methods using multiple gridded products are able to attain, on average, a 5 W/m 2 reduction in continuous ranked probability score than that leveraging just a single product.
Groundwater contamination sources identification based on the Long-Short Term Memory network

When simulation–optimization (S/O) is used for groundwater contamination source identification (GCSI), to reduce the calculation load and time generated by calling the simulation model, a surrogate model is often used instead of the simulation model. However, when the conversion relationship between the simulation model inputs and outputs is complex and the model is highly nonlinear, the commonly used approach of building a surrogate model may lose its advantage. This study use and check a deep learning method with the long-short term memory (LSTM) network which has great potential for characterizing the input–output conversion relationship of complex nonlinear numerical simulations, to a surrogate model of the simulation model. The accuracy of the surrogate model developed by the LSTM method was compared with that of commonly used surrogate models developed by the Kriging method, the radial basis function network method, and the kernel extreme learning machine method. The surrogate model with the highest accuracy was linked to the optimization model, and the optimization model was solved to identify the contamination source information. Results show that compared with the other three methods, the surrogate model constructed by the LSTM method had the best accuracy and generalization performance. LSTM is therefore an effective method of building a surrogate model. Linking the LSTM surrogate model to an optimization model and then solving the optimization model can save approximately 99% of the computing load and time otherwise required.
Identifying and Mitigating Phishing Attack Threats in IoT Use Cases Using a Threat Modelling Approach

Internet of things (IoT) is a technology that enables our daily life objects to connect on the Internet and to send and receive data for a meaningful purpose. In recent years, IoT has led to many revolutions in almost every sector of our society. Nevertheless, security threats to IoT devices and networks are relentlessly disruptive, because of the proliferation of Internet technologies. Phishing is one of the most prevalent threats to all Internet users, in which attackers aim to fraudulently extract sensitive information of a user or system, using fictitious emails, websites, etc. With the rapid increase in IoT devices, attackers are targeting IoT devices such as security cameras, smart cars, etc., and perpetrating phishing attacks to gain control over such vulnerable devices for malicious purposes. In recent decades, such scams have been spreading, and they have become increasingly advanced over time. By following this trend, in this paper, we propose a threat modelling approach to identify and mitigate the cyber-threats that can cause phishing attacks. We considered two significant IoT use cases, i.e., smart autonomous vehicular system and smart home. The proposed work is carried out by applying the STRIDE threat modelling approach to both use cases, to disclose all the potential threats that may cause a phishing attack. The proposed threat modelling approach can support the IoT researchers, engineers, and IoT cyber-security policymakers in securing and protecting the potential threats in IoT devices and systems in the early design stages, to ensure the secure deployment of IoT devices in critical infrastructures.
Turning Frequency to Resolution: Video Super-resolution via Event Cameras

State-of-the-art video super-resolution (VSR) methods focus on exploiting inter- and intra-frame correlations to estimate high-resolution (HR) video frames from low-resolution (LR) ones. In this paper, we study VSR from an exotic perspective, by explicitly looking into the role of temporal frequency of video frames. Through experiments, we observe that a higher frequency, and hence a smaller pixel displacement between consecutive frames, tends to de-liver favorable super-resolved results. This discovery motivates us to introduce Event Cameras, a novel sensing de-vice that responds instantly to pixel intensity changes and produces up to millions of asynchronous events per second, to facilitate VSR. To this end, we propose an Event-based VSR framework (E-VSR), of which the key component is an asynchronous interpolation (EAI) module that reconstructs a high-frequency (HF) video stream with uniform and tiny pixel displacements between neighboring frames from an event stream. The derived HF video stream is then encoded into a VSR module to recover the desired HR videos. Furthermore, an LR bi-directional interpolation loss and an HR self-supervision loss are also introduced to respectively regulate the EAI and VSR modules. Experiments on both real-world and synthetic datasets demonstrate that the proposed approach yields results superior to the state of the art.
Weakly Supervised Temporal Adjacent Network for Language Grounding

Temporal language grounding (TLG) is a fundamental and challenging problem for vision and language understanding. Existing methods mainly focus on fully supervised setting with temporal boundary labels for training, which, however, suffers expensive cost of annotation. In this work, we are dedicated to weakly supervised TLG, where multiple description sentences are given to an untrimmed video without temporal boundary labels. In this task, it is critical to learn a strong cross-modal semantic alignment between sentence semantics and visual content. To this end, we introduce a novel weakly supervised temporal adjacent network (WSTAN) for temporal language grounding. Specifically, WSTAN learns cross-modal semantic alignment by exploiting temporal adjacent network in a multiple instance learning (MIL) paradigm, with a whole description paragraph as input. Moreover, we integrate a complementary branch into the framework, which explicitly refines the predictions with pseudo supervision from the MIL stage. An additional self-discriminating loss is devised on both the MIL branch and the complementary branch, aiming to enhance semantic discrimination by self-supervising. Extensive experiments are conducted on three widely used benchmark datasets, <i>i.e.</i>, ActivityNet-Captions, Charades-STA, and DiDeMo, and the results demonstrate the effectiveness of our approach.
ChaLearn LAP Large Scale Signer Independent Isolated Sign Language Recognition Challenge: Design, Results and Future Research

The performances of Sign Language Recognition (SLR) systems have improved considerably in recent years. However, several open challenges still need to be solved to allow SLR to be useful in practice. The research in the field is in its infancy in regards to the robustness of the models to a large diversity of signs and signers, and to fairness of the models to performers from different demographics. This work summarises the ChaLearn LAP Large Scale Signer Independent Isolated SLR Challenge, organised at CVPR 2021 with the goal of overcoming some of the aforementioned challenges. We analyse and discuss the challenge design, top winning solutions and suggestions for future research. The challenge attracted 132 participants in the RGB track and 59 in the RGB+Depth track, receiving more than 1.5K submissions in total. Participants were evaluated using a new large-scale multi-modal Turkish Sign Language (AUTSL) dataset, consisting of 226 sign labels and 36,302 isolated sign video samples performed by 43 different signers. Winning teams achieved more than 96% recognition rate, and their approaches benefited from pose/hand/face estimation, transfer learning, external data, fusion/ensemble of modalities and different strategies to model spatio-temporal information. However, methods still fail to distinguish among very similar signs, in particular those sharing similar hand trajectories.
Quick Response (QR) codes for patient information delivery: A digital innovation during the coronavirus pandemic

Over the past year, Quick Response (QR) codes have played a significant role in our day-to-day lives in reducing the transmission and tracking the spread of COVID-19. In this article, we share our innovation utilising QR codes to replace paper information leaflets allowing patients to immediately access the required information on their own personal device. This is contactless and therefore preferred to reduce viral transmission, as well as having several other advantages. Our findings demonstrate that QR codes are a familiar, easy-to-use system and a preferred tool for delivering patient information over paper leaflets. The findings and methodology may be of benefit to other units seeking to improve their infection control in the COVID-19 era.
Novel Hopfield Neural Network Model with Election Algorithm for Random 3 Satisfiability

One of the influential models in the artificial neural network (ANN) research field for addressing the issue of knowledge in the non-systematic logical rule is Random k Satisfiability. In this context, knowledge structure representation is also the potential application of Random k Satisfiability. Despite many attempts to represent logical rules in a non-systematic structure, previous studies have failed to consider higher-order logical rules. As the amount of information in the logical rule increases, the proposed network is unable to proceed to the retrieval phase, where the behavior of the Random Satisfiability can be observed. This study approaches these issues by proposing higher-order Random k Satisfiability for k ≤ 3 in the Hopfield Neural Network (HNN). In this regard, introducing the 3 Satisfiability logical rule to the existing network increases the synaptic weight dimensions in Lyapunov’s energy function and local field. In this study, we proposed an Election Algorithm (EA) to optimize the learning phase of HNN to compensate for the high computational complexity during the learning phase. This research extensively evaluates the proposed model using various performance metrics. The main findings of this research indicated the compatibility and performance of Random 3 Satisfiability logical representation during the learning and retrieval phase via EA with HNN in terms of error evaluations, energy analysis, similarity indices, and variability measures. The results also emphasized that the proposed Random 3 Satisfiability representation incorporates with EA in HNN is capable to optimize the learning and retrieval phase as compared to the conventional model, which deployed Exhaustive Search (ES).
Supervoxel Convolution for Online 3D Semantic Segmentation

Online 3D semantic segmentation, which aims to perform real-time 3D scene reconstruction along with semantic segmentation, is an important but challenging topic. A key challenge is to strike a balance between efficiency and segmentation accuracy. There are very few deep-learning-based solutions to this problem, since the commonly used deep representations based on volumetric-grids or points do not provide efficient 3D representation and organization structure for online segmentation. Observing that on-surface supervoxels, i.e., clusters of on-surface voxels, provide a compact representation of 3D surfaces and brings efficient connectivity structure via supervoxel clustering, we explore a supervoxel-based deep learning solution for this task. To this end, we contribute a novel convolution operation (SVConv) directly on supervoxels. SVConv can efficiently fuse the multi-view 2D features and 3D features projected on supervoxels during the online 3D reconstruction, and leads to an effective supervoxel-based convolutional neural network, termed as Supervoxel-CNN , enabling 2D-3D joint learning for 3D semantic prediction. With the Supervoxel-CNN , we propose a clustering-then-prediction online 3D semantic segmentation approach. The extensive evaluations on the public 3D indoor scene datasets show that our approach significantly outperforms the existing online semantic segmentation systems in terms of efficiency or accuracy.
Verifying learning-augmented systems

The application of deep reinforcement learning (DRL) to computer and networked systems has recently gained significant popularity. However, the obscurity of decisions by DRL policies renders it hard to ascertain that learning-augmented systems are safe to deploy, posing a significant obstacle to their real-world adoption. We observe that specific characteristics of recent applications of DRL to systems contexts give rise to an exciting opportunity: applying formal verification to establish that a given system provably satisfies designer/user-specified requirements, or to expose concrete counter-examples. We present whiRL, a platform for verifying DRL policies for systems, which combines recent advances in the verification of deep neural networks with scalable model checking techniques. To exemplify its usefulness, we employ whiRL to verify natural equirements from recently introduced learning-augmented systems for three real-world environments: Internet congestion control, adaptive video streaming, and job scheduling in compute clusters. Our evaluation shows that whiRL is capable of guaranteeing that natural requirements from these systems are satisfied, and of exposing specific scenarios in which other basic requirements are not.
Light Field Image Quality Assessment With Auxiliary Learning Based on Depthwise and Anglewise Separable Convolutions

In multimedia broadcasting, no-reference image quality assessment (NR-IQA) is used to indicate the user-perceived quality of experience (QoE) and to support intelligent data transmission while optimizing user experience. This paper proposes an improved no-reference light field image quality assessment (NR-LFIQA) metric for future immersive media broadcasting services. First, we extend the concept of depthwise separable convolution (DSC) to the spatial domain of light field image (LFI) and introduce “light field depthwise separable convolution (LF-DSC)”, which can extract the LFI’s spatial features efficiently. Second, we further theoretically extend the LF-DSC to the angular space of LFI and introduce the novel concept of “light field anglewise separable convolution (LF-ASC)”, which is capable of extracting both the spatial and angular features for comprehensive quality assessment with low complexity. Third, we define the spatial and angular feature estimations as auxiliary tasks in aiding the primary NR-LFIQA task by providing spatial and angular quality features as hints. To the best of our knowledge, this work is the first exploration of deep auxiliary learning with spatial-angular hints on NR-LFIQA. Experiments were conducted in mainstream LFI datasets such as Win5-LID and SMART with comparisons to the mainstream full reference IQA metrics as well as the state-of-the-art NR-LFIQA methods. The experimental results show that the proposed metric yields overall 42.86% and 45.95% smaller prediction errors than the second-best benchmarking metric in Win5-LID and SMART, respectively. In some challenging cases with particular distortion types, the proposed metric can reduce the errors significantly by more than 60%.
A Qualitative Similarity Framework for the Interpretation of Natural Language Similarity Expressions

Abstract In this paper, a representational framework is presented featuring a qualitative notion of similarity. It is aimed at issues of natural language semantics, in particular the semantics of expressions of similarity and sameness and their role in comparison and ad-hoc kind formation. The framework makes use of attribute spaces, which are well-established in AI and also in some branches of natural language semantics, e.g., frame-based approaches (Barsalou 1992). What distinguishes attribute spaces and representations as proposed in this paper is the idea of systems of predicates on attribute spaces corresponding to predicates on the domain. On the worldy side, a domain includes a set of relevant predicates talking about individuals. These predicates have counterparts on the representational side talking about points of an attribute space. Counterpart predicates are required to be consistent with their originals; more precisely, they have to agree in truth-value on the set of positive and negative exemplars thereby approximating the original predicates. Moreover, counterpart predicates will be assumed to have convex and open extensions. This system facilitates a qualitative notion of similarity which is suited to account for the meaning of natural language similarity expressions and, furthermore, their role in comparison and ad-hoc kind formation.
Dilated densely connected U-Net with uncertainty focus loss for 3D ABUS mass segmentation

Accurate segmentation of breast mass in 3D automated breast ultrasound (ABUS) images plays an important role in qualitative and quantitative ABUS image analysis. Yet this task is challenging due to the low signal to noise ratio and serious artifacts in ABUS images, the large shape and size variation of breast masses, as well as the small training dataset compared with natural images. The purpose of this study is to address these difficulties by designing a dilated densely connected U-Net (D2U-Net) together with an uncertainty focus loss. A lightweight yet effective densely connected segmentation network is constructed to extensively explore feature representations in the small ABUS dataset. In order to deal with the high variation in shape and size of breast masses, a set of hybrid dilated convolutions is integrated into the dense blocks of the D2U-Net. We further suggest an uncertainty focus loss to put more attention on unreliable network predictions, especially the ambiguous mass boundaries caused by low signal to noise ratio and artifacts. Our segmentation algorithm is evaluated on an ABUS dataset of 170 volumes from 107 patients. Ablation analysis and comparison with existing methods are conduct to verify the effectiveness of the proposed method. Experiment results demonstrate that the proposed algorithm outperforms existing methods on 3D ABUS mass segmentation tasks, with Dice similarity coefficient, Jaccard index and 95% Hausdorff distance of 69.02%, 56.61% and 4.92 mm, respectively. The proposed method is effective in segmenting breast masses on our small ABUS dataset, especially breast masses with large shape and size variations.