Reviewerly/exHarmony huggingface dataset:
A Wireless Wearable Sensor for Pointing and Arm-Gesture Recognition

A low-power, wireless, hand-mounted, pointing and arm gesture recognition tool has been designed for convenient, screen-free, symbolic communication for mobile robots in the field. The device is based on commercially-available integrated sensors (accelerometer, gyroscope, and magnetometer), an IoT-enabled microcontroller, and 3-D printed parts. The design of the gesture set dramatically simplifies recognition and operator training and reduces power consumption. Guided by cognitive load and power considerations, the device is triggered by finger switches but does not significantly impede other manual tasks. We describe and analyze the design of the device and gestures and demonstrate the recognition performance on a set of 12 gestures using 3 test subjects. Easily calibrated, trained, and reconfigured, the system can achieve essentially 100% recognition when operating on “self”-trained gestures.
Deep Learning Assists Surveillance Experts: Toward Video Data Prioritization

Video summarization (VS) suppresses high-dimensional (HD) video data by only extracting the important information. However, prior research has not focused on the need for surveillance VS, that is used for many applications to assist video surveillance experts, including video retrieval and data storage. In addition, mainstream techniques commonly use two-dimensional (2-D) deep models for VS, ignoring event occurrences. Accordingly, we present a two-fold 3-D deep learning-assisted VS framework. First, we employ an inflated 3-D ConvNet model to extract temporal features; these features are optimized using a proposed encoder mechanism. The input video is temporally segmented using a feature comparison technique for selecting a single frame from each video segment. The segmented shots are evaluated using our novel shot segmentation evaluation scheme and are input into a saliency computation mechanism for keyframe selection in a second fold. Qualitative and quantitative analyses over VS benchmarks and surveillance videos demonstrate the superior performance of our framework, with 0.3- and 4.2-unit increases in the F1 scores for YouTube and title-based video summarization datasets, respectively. Along with accurate VS, a key contribution of our study is the novel shot segmentation criterion prior to VS, which can be used as a benchmark in future research to effectively prioritize HD visual data.
Facial Expression Algorithm using Attention Mechanism

Facial expression recognition is a basic task in computer vision, which has wide applications, such as human interaction, image recognition, etc. Numerous frameworks have been proposed for better accuracy and speed. In this work the author aim to exploit the effectiveness of visual attention mechanism to this task and investigate classical convolutional neural networks models for large scale object classification and have applied two prominent models VGG-16(very deep neural networks) network and SENet(Squeeze-and-Excitation Networks) with dataset in Kaggle in the field of facial expressions classification to better explore their accuracy. Cause VGG16 do have some defaults, SE block in SENet is added in VGG16 for better accuracy. The experiment on FER2013 dataset validates the effectiveness of the method proposed as before. Our method provides a comprehensive analysis for exploiting the visual attention modules to facial expression recognition. Our proposed algorithm can be equipped into other popular backbone to achieve better performance.
Medical Image Segmentation Based on Recurrent Residual Convolution Neural Network

In order to solve the problem of network degradation when using deep learning for medical image segmentation, and to achieve more accurate image segmentation effect by extracting larger scale feature mapping, an image segmentation model based on recurrent residual convolutional neural network is proposed. Firstly, the recurrent convolution unit is introduced to realize the feature extraction on discrete step size, improves the utilization of image context semantic information, and realizes more extensive feature mapping extraction. Then, combined with the residual learning unit and the recurrent convolution unit, the recurrent residual convolution unit is formed to replace the feed-forward convolution unit of the ordinary convolution neural network to solve the network degradation problem faced by the deep network model. Finally, the full-scale skip connection is introduced to fuse the feature images under different scales to generate the segmented image. The experimental results of three datasets compared with the other four algorithms in PyTorch environment show that the proposed algorithm has better segmentation performance and higher image segmentation accuracy.
Research on the Construction of Graphical Data Intelligence Education Driven by Digital Intelligence Integration

With the development and application of technologies such as 5G and big data artificial intelligence, humanity has entered the era of digital intelligence convergence (BD + AI), which has enabled the future of education and learning to move into a new era driven by graphical data. In the field of education, graphical data is intuitive, panoramic, interactive, intelligent, scalable and narrative, bringing effective experiential aspects to student learning, teacher teaching, learning assessment and teaching management. This paper discusses the application of digital intelligence to education and teaching with the help of its multi-technology fusion model, the research model of knowledge mapping, and the application of intelligent education as pictorial data that we can perceive and analyse, so that intelligent graphical data, data-driven, and data empowerment can provide better and more data experiences and services for teaching and learning in education.
Remote Guided Robotics via Leap Motion and Mixed Reality

The global teleoperation and telerobotics market is projected to reach $98.3billion (USD) by 2027. With the benefits of 5G coverage almost anywhere in the world will allow teleoperated robots to be implemented globally via the cloud. Remote Guided Robotics has excellent potential in existing markets, attempting to remove operators from dangerous tasks and providing access to labour in remote areas, where labour is scarce. This research introduces a configuration of technology that reduces the knowledge gap of what is required to successfully achieve high accuracy and realistic control of a robot in real-time. The system implemented in this paper utilises an infrared camera from Leap Motion to capture and track hand gestures and movements, a Kuka 6 DOF robot and software packages to follow the hand movements from the UltraLeap Leap Motion Controller (LMC) in real-time, and an HTC VR headset for a mixed reality experience where two cameras are placed in the robot work-cell to send back a 3D stereoscopic image to allow the operator to have an immersed experience when remotely controlling the robot. The system proved to have a repeatability of approx. <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\pm \mathbf{10}\mathbf{mm}$</tex> , and an accuracy of 0.1mm. The gripper was able to successfully pick up 86% of the time, with a final test of gripping an item and guiding it through a bandsaw at an average rate of 24.2 seconds. Testings and simulations completed in this paper outline the level of accuracy and repeatability of the remote guide robotic system. This teleoperated robotic system has the ability to protect operators' safety and meet the explosive growth of needs in the various industries.
Spoof Speech Detection Based on Raw Cross-Dimension Interaction Attention Network

Benefiting from advances in speech synthesis and speech conversion technology, artificial speech is so close to natural speech that it is sensory indistinguishable. This situation brings great challenges to the security of voice-based biometric authentication systems. In this work, we propose an end-to-end spoofing detection method which first augments the raw-audio waveform with random channel masking, then feeds it into the lightweight spectral-temporal attention module for cross-dimensional interaction, and finally selects an appropriate attention fusion method to maximise the potential of capturing interactive cues in both spectral and temporal domains. The experimental results show that the proposed method can effectively improve the accuracy of spoof speech detection.
COMPOSV++: Light Weight Online Signature Verification Framework Through Compound Feature Extraction and Few-Shot Learning

Online Signature Verification (OSV) is a systematically used biometric characteristic to endorse the genuineness of a user to access real time applications like healthcare, m-payment, etc. Because OSV frameworks are used in real-time applications and it is difficult to acquire a sufficient number of signature samples from users, they must meet a critical requirement: they must be able to detect skilled and random signature presentation attacks effectively with fewer training signature samples and a faster response time. To meet these needs, we developed a depth wise separable (DWS) convolution-based OSV framework that realizes one/few shot learning in inference phase. In addition to it, we have designed a compound feature extraction technique, which extracts maximum seven features from a set of 100 features in MCYT-100, and 3 features from a set of 47 in case of {SVC, SUSIG} datasets. The framework uses only three to seven features per signature to resist the signature presentation attacks. We have extensively evaluated our framework, by performing thorough experiments with three datasets i.e. MCYT-100, SVC and SUSIG. The model results state of the art EER in all skilled categories of SVC and SUSIG datasets.
Die unerträgliche Wichtigkeit eines Kontextes

Of course, the title of this paper should evoke a book by Milan Kundera The unbearable lightness of being which is in English (as well as German) a word-by-word translation of the original title Nesnesitelna lehkost byti, There are, however, not many such linear translations of labels like these.
Application and Development of Intelligent Translation Technology

Intelligent translation is the product of the combination of artificial intelligence technology and machine translation technology, and is the product of the development of artificial intelligence, which greatly enhances the accuracy of machine translation. Its application will be greatly expanded, not only as an intelligent auxiliary tool for human translation. It can also be used in voice services, robotics, etc. Although there are still certain limitations such as insufficient training corpus, insufficient recognition of context, lack of creativity, etc. In the future, with the continuous expansion of artificial intelligence, it will continue to overcome its weaknesses in all aspects and eventually replace human translation.
Transparent and Tamper-Proof Event Ordering in the Internet of Things Platforms

Today, the audit and diagnosis of the causal relationships between the events in a trigger-action-based event chain (e.g., why is a light turned on in a smart home?) in the Internet of Things (IoT) platforms are untrustworthy and unreliable. The current IoT platforms lack techniques for transparent and tamper-proof ordering of events due to their device-centric logging mechanism. In this article, we develop a framework that facilitates tamper-proof transparency and event order in an IoT platform by proposing a Blockchain protocol and adopting the vector clock system, both tailored for the resource-constrained heterogeneous IoT devices, respectively. To cope with the unsuited storage (e.g., ledger) and computing power (e.g., proof-of-work puzzle) requirements of the Blockchain in the commercial off-the-shelf IoT devices, we propose a partial consistent cut protocol and engineer a modular arithmetic-based lightweight proof of work puzzle, respectively. To the best of our knowledge, this is the first Blockchain designed for resource-constrained heterogeneous IoT platforms. Our event ordering protocol based on the vector clock system is also novel for the IoT platforms. We implement our framework using an IoT gateway and 30 IoT devices. We experiment with ten concurrent trigger-action-based event chains while each chain involves 20 devices, and each device participates in five different chains. The results show that our framework may order these events in 2.5 s while consuming approximately 140 mJ of energy per device. The results, hence, demonstrate the proposed platform as a practical choice for many IoT applications, such as smart home, traffic monitoring, and crime investigation.
Language-Attention Modular-Network for Relational Referring Expression Comprehension in Videos

Referring expression (RE) for video domain describes the video using a natural language expression. Relational RE comprehension in a video domain localizes an object in relation to a distinguishing context object. Unlike object grounding in videos using REs, not much work has been done in videos using relational REs. In this paper, we focus on (1) relational RE comprehension for videos, and (2) demonstrating the significance of attention for the task. We propose a novel modular network based approach for relational RE comprehension in highly ambiguous settings for videos. We show the significance of the language attention in modular approach by: (1) Using two different networks, i.e., modATN consisting of attention mechanism, visual modules, and a natural language expression input, and modSTR consisting of visual modules, and structured input (subject, subject adjective, object, object adjective, action, relation); (2) Introducing a new dataset having structured RE for relation RE comprehension task in modSTR. Finally, we propose an optimised modular network that outperforms and shows significant improvements over the baseline networks.
Information Technology, Business Sustainability and Female Economic Participation in Sub-Saharan Africa

This study assesses how business/financial sustainability in the perspective of financial stability moderates information technology to influence female economic participation in 49 countries in Sub-Saharan Africa for the period 2008-2018. The empirical evidence is based on Tobit regressions that enabled the study to account for the censored nature of the outcome variables. The following important findings are established. First, ICT dynamics (mobile phone penetration, internet penetration and fixed broadband subscriptions) are consistently moderated by business sustainability to positively affect female employment in the industry. Second, business sustainability scores need to exceed certain thresholds before moderating fixed broadband subscriptions to induce favorable overall effects on female employment, female labour force participation and female unemployment rates. These thresholds are 18.742 and 19.505 Z-scores for positive effects on female employment and female labour force participation, respectively and a Z-score of 17.300 for a negative impact on female unemployment. The thresholds which should be exceeded are within policy reach, make economic sense and are policy relevant. The study contributes to the extant literature by providing actionable thresholds of business sustainability that can be employed by policy makers in order for information technology to positively influence female economic inclusion in Sub-Saharan Africa.
Learning Semantic Keypoints for Object Detection in Aerial Images

Object detection in aerial images has achieved remarkable progress with the advent of deep convolutional neural networks (CNNs). It is, however, still a challenging task since the objects in aerial images are arbitrarily oriented and often densely packed. In this letter, we propose a novel method for oriented object detection in aerial images that represents objects as rotation equivariant semantic keypoints. Unlike conventional methods that represent object rotation according to angles from each axis in the Cartesian coordinate system, we represent object using a canonical orientation to ensure rotation equivariance. We accomplish this by representing an object as semantic keypoints, where each keypoint of the object consistently corresponds to the semantic part, regardless of rotation variation. To this end, we define the “head” point of the object as the canonical orientation and the remaining bounding box vectors as semantic keypoints in clockwise order. To discriminate visual attributes between different categories, we further use category-specific semantic keypoints, so that object classification and localization can be jointly solved in a cooperative manner. Our experiments demonstrate the effectiveness of rotation equivariant semantic keypoints on oriented object detection.
RepoQuester: A Tool Towards Evaluating GitHub Projects

Given the drastic rise of repositories on GitHub, it is often hard for developers to find relevant projects meeting their requirements as analyzing source code and other artifacts is effort-intensive. In our prior work, we proposed Repo Reaper (or simply Reaper) that assesses GitHub projects based on seven metrics spanning across project collaboration, quality, and maintenance. Reaper identified 1.4 million projects out of nearly 1.8 million projects to have no purpose for collaboration or software development by classifying them into ‘engineered’ and ‘non-engineered’ software projects. While Reaper can be used to assess millions of repositories based on GHTorrent, it is not designed to be used by developers for standalone repositories on local machines and is dependent on GHTorrent. Hence, in this paper, we propose a re-engineered and extended command-line tool named RepoQuester that aims to assist developers in evaluating GitHub projects on their local machines. RepoQuester computes metrics for projects and does not classify projects into ‘engineered’ and ‘non-engineered’ ones. However, to demonstrate the correctness of metric scores produced by RepoQuester, we have performed the project classification on the Reaper’s training and validation datasets by updating them with the latest metric scores (as reported by RepoQuester). These datasets have their ground truth manually established. During the analysis, we observed that the machine learning classifiers built on the updated datasets produced an F1 score of 72%. During the evaluation, for each project, we found that RepoQuester can analyze metric scores in less than 10 seconds. A demo video explaining the tool highlights and usage is available at https://youtu.be/Q8OdmNzUfN0, and source code at https://github.com/Kowndinya2000/Repoquester.
Quality assurance of generative dialog models in an evolving conversational agent used for Swedish language practice

Due to the migration megatrend, efficient and effective second-language acquisition is vital. One proposed solution involves AI-enabled conversational agents for person-centered interactive language practice. We present results from ongoing action research targeting quality assurance of proprietary generative dialog models trained for virtual job interviews. The action team elicited a set of 38 requirements for which we designed corresponding automated test cases for 15 of particular interest to the evolving solution. Our results show that six of the test case designs can detect meaningful differences between candidate models. While quality assurance of natural language processing applications is complex, we provide initial steps toward an automated framework for machine learning model selection in the context of an evolving conversational agent. Future work will focus on model selection in an MLOps setting.
SoftFusion: A Low-Cost Approach to Enhance Reliability of Object Detection Applications

DNNs are deployed in mission critical applications such as object detection in autonomous vehicles (AVs). Ensuring the reliability of detections made by these models plays an essential role in robustness and practicality of these systems. By analyzing traditional object detectors, we observe a significant number of objects being missed by the detector, which is a problem for safe deployment of AVs. Traditionally, sensor fusion wherein detections from multiple sensors (camera, LiDAR, etc.) are combined to create a robust detection system is utilized, but suffers from high cost and computing overheads. To attack this seemingly necessary tradeoff, we propose SoftFusion in this paper. SoftFusion introduces the concept of intra-sensor fusion wherein a diversity of inputs is synthesized during runtime and can be efficiently evaluated by the original object detector. The predictions are then intelligently combined to realize robust detections over the baseline detector but at a fraction of the cost of traditional fusion. SoftFusion is evaluated across 7 different benchmarks over 2 different datasets, and is shown to achieve 3.45% gain in average precision with less than 23% latency overhead. As context for these results, prior research demonstrates that hard fusion of camera and LiDAR provides on average 4.02% gain in average precision in similar tasks with an overhead of approximately up to 2× computation.
A Privacy-Preserving Federated Learning with Mutual Verification on Vector Spaces

Federated learning has received widespread attention in recent years, since it trains a model by only sharing gradients without accessing training sets. In this paper, we consider two security issues in the training process of federated learning, i.e., privacy preservation and message verification, which mainly consider the security of the local gradients uploaded by clients and the aggregation result. We give the detail design about the privacy preserving federated learning with mutual authentication, which provides the privacy-preserving and mutually verifiable federated learning framework on the vector space. To extend the numerical operations to the vector space, we modify the secret sharing of numbers to that of vectors, and advance the commitment to numbers to a commitment to polynomials.
Face Image Inpainting Using Attribute Guided

Face image inpainting is becoming more and more significant in the practical fields of criminal investigation, cosmetic plastic surgery, and security protection. But most of traditional existing work is based on diffusion and texture methods, failing to effectively use the embedding attribute knowledge and semantic information of face images. It is difficult to ensure the authenticity and preserve semantic consistency at the same time. To tackle this problem, a face image inpainting method based on attribute control is proposed. First, a modified generative adversarial network is proposed to restore face images and control facial features by adding additional face attribute information and a classification network. Then an attention module based on multi-scale feature fusion is proposed, through increasing the receptive field of the neural network to obtain more global features to improve face image inpainting quality without increasing the arithmetic complexity. Finally, combining the four evaluation metrics, MAE, PSNR, SSIM, and FID, the experiments on the CelebA dataset show that the proposed method can not only generate high-quality face image inpainting results steered by semantic information, but also generate corresponding facial features based on attribute control.