Reviewerly/exHarmony huggingface dataset:
Beyond blur

To peripheral vision, a pair of physically different images can look the same. Such pairs are metamers relative to each other, just as physically-different spectra of light are perceived as the same color. We propose a real-time method to compute such ventral metamers for foveated rendering where, in particular for near-eye displays, the largest part of the framebuffer maps to the periphery. This improves in quality over state-of-the-art foveation methods which blur the periphery. Work in Vision Science has established how peripheral stimuli are ventral metamers if their statistics are similar. Existing methods, however, require a costly optimization process to find such metamers. To this end, we propose a novel type of statistics particularly well-suited for practical real-time rendering: smooth moments of steerable filter responses. These can be extracted from images in time constant in the number of pixels and in parallel over all pixels using a GPU. Further, we show that they can be compressed effectively and transmitted at low bandwidth. Finally, computing realizations of those statistics can again be performed in constant time and in parallel. This enables a new level of quality for foveated applications such as such as remote rendering, level-of-detail and Monte-Carlo denoising. In a user study, we finally show how human task performance increases and foveation artifacts are less suspicious, when using our method compared to common blurring.
Secret Sharing-based Personal Health Records Management for the Internet of Health Things

The holistic concept of smart cities has been adopted to increase economic, environmental, and social sustainability. For the sustainable health of a smart city inhabitant, there is a requirement for innovations in implementing health solutions. Given the exponential growth of the Internet of Health Things (IoHT) in health infrastructures, bulk amounts of health-related data are accumulated in Personal Health Records (PHR) and processed in storage data centers. One of the major challenges in this scenario is the security of this enormous medical data. The conventional encryption schemes generate single cipher images making the cryptosystem vulnerable to single-point attacks. This paper evaluates a distributed security module for the clinical images that form 80% of the health data. We utilize the Rivest Cipher 6 (RC6) encryption algorithm with a Computational Secret Sharing Scheme (CSIS) for distributed storage of clinical images. The key is shared using a Perfect Secret Sharing (PSS) scheme. We show that (k-1) key-shares and the 'n' image shares can be made public because of the perfect security of the scheme. The rest of the key shares can be secured using the Deoxyribonucleic Acid substitution (DNA substitution) method. Analysis of the shares generated reveals the strength of the cryptosystem and gives an insight into the degree of security provided to the health-related data. The entropy of the generated shares is higher than 7.99, and structural similarity (SSIM) values are negligible. The Number of Pixels Change Rate (NPCR) values are greater than 99.55% for all the shares that show a high diffusion measure. Comparison with state-of-the-art schemes reveals and validates the robustness of the cryptosystem.
Child Drawing Development Optimization Algorithm Based on Child’s Cognitive Development

This paper proposes a novel metaheuristic Child Drawing Development Optimization (CDDO) algorithm inspired by the child's learning behaviour and cognitive development using the golden ratio to optimize the beauty behind their art. The golden ratio was first introduced by the famous mathematician Fibonacci. The ratio of two consecutive numbers in the Fibonacci sequence is similar, and it is called the golden ratio, which is prevalent in nature, art, architecture, and design. CDDO uses golden ratio and mimics cognitive learning and child's drawing development stages starting from the scribbling stage to the advanced pattern-based stage. Hand pressure width, length and golden ratio of the child's drawing are tuned to attain better results. This helps children with evolving, improving their intelligence and collectively achieving shared goals. CDDO shows superior performance in finding the global optimum solution for the optimization problems tested by 19 benchmark functions. Its results are evaluated against more than one state of art algorithms such as PSO, DE, WOA, GSA, and FEP. The performance of the CDDO is assessed, and the test result shows that CDDO is relatively competitive through scoring 2.8 ranks. This displays that the CDDO is outstandingly robust in exploring a new solution. Also, it reveals the competency of the algorithm to evade local minima as it covers promising regions extensively within the design space and exploits the best solution.
Contrastive Self-supervised Learning for Sensor-based Human Activity Recognition

Deep Learning models, applied to a sensor-based Human Activity Recognition task, usually require vast amounts of annotated time-series data to extract robust features. However, annotating signals coming from wearable sensors can be a tedious and, often, not so intuitive process, that requires specialized tools and predefined scenarios, making it an expensive and time-consuming task. This paper combines one of the most recent advances in Self-Supervised Leaning (SSL), namely a SimCLR framework, with a powerful transformer-based encoder to introduce a Contrastive Self-supervised learning approach to Sensor-based Human Activity Recognition (CSSHAR) that learns feature representations from unlabeled sensory data. Extensive experiments conducted on three widely used public datasets have shown that the proposed method outperforms recent SSL models. Moreover, CSSHAR is capable of extracting more robust features than the identical supervised transformer when transferring knowledge from one dataset to another as well as when very limited amounts of annotated data are available.
Detecting ethnicity-targeted hate speech in Russian social media texts

Ethnicity-targeted hate speech has been widely shown to influence on-the-ground inter-ethnic conflict and violence, especially in such multi-ethnic societies as Russia. Therefore, ethnicity-targeted hate speech detection in user texts is becoming an important task. However, it faces a number of unresolved problems: difficulties of reliable mark-up, informal and indirect ways of expressing negativity in user texts (such as irony, false generalization and attribution of unfavored actions to targeted groups), users' inclination to express opposite attitudes to different ethnic groups in the same text and, finally, lack of research on languages other than English. In this work we address several of these problems in the task of ethnicity-targeted hate speech detection in Russian-language social media texts. This approach allows us to differentiate between attitudes towards different ethnic groups mentioned in the same text – a task that has never been addressed before. We use a dataset of over 2,6M user messages mentioning ethnic groups to construct a representative sample of 12K instances (ethnic group, text) that are further thoroughly annotated via a special procedure. In contrast to many previous collections that usually comprise extreme cases of toxic speech, representativity of our sample secures a realistic and, therefore, much higher proportion of subtle negativity which additionally complicates its automatic detection. We then experiment with four types of machine learning models, from traditional classifiers such as SVM to deep learning approaches, notably the recently introduced BERT architecture, and interpret their predictions in terms of various linguistic phenomena. In addition to hate speech detection with a text-level two-class approach (hate, no hate), we also justify and implement a unique instance-based three-class approach (positive, neutral, negative attitude, the latter implying hate speech). Our best results are achieved by using fine-tuned and pre-trained RuBERT combined with linguistic features, with F1-hate=0.760, F1-macro=0.833 on the text-level two-class problem comparable to previous studies, and F1-hate=0.813, F1-macro=0.824 on our unique instance-based three-class hate speech detection task. Finally, we perform error analysis, and it reveals that further improvement could be achieved by accounting for complex and creative language issues more accurately, i.e., by detecting irony and unconventional forms of obscene lexicon.
“Do You Think You Have What it Takes?” – Exploring Predictors of Pre-Service Teachers’ Prospective ICT Use

Abstract Despite extensive efforts to support teachers with the integration of information and communication technologies (ICT) into their classroom practice, current research reports that teachers face immense challenges when integrating ICT into their teaching. This issue has become even more relevant with the rapid spread of the COVID-19 virus, forcing schools around the world to close for an indefinite period of time and thus to offer remote digital learning solutions. Against this background, this study focused on examining the predictors of pre-service teachers’ prospective ICT use and investigated the heterogeneous results of previous research related to ICT use and gender. Following the ‘will, skill, tool’ framework, the study examined relevant factors of pre-service teachers’ ( N = 103) prospective ICT use for teaching and learning processes by means of multiple regression analyses. The analyses included pre-service teachers’ background characteristics, ICT profiles (attitudes and self-efficacy), digital competencies and use of digital tools in order to explore their role in future in-class use of ICT. They also show that there are no gender differences in pre-service teachers’ prospective ICT integration. However, male pre-service teachers hold more positive attitudes towards ICT use than their female counterparts. Additionally, the findings reveal that the two strongest predictors of pre-service teachers’ future ICT use are their attitudes and perceived competency to teach and implement technology in their teaching practices. Finally, the results provide important information about teachers’ training needs. Implications of the results and further research are discussed.
Intelligent blockchain management for distributed knowledge graphs in IoT 5G environments

Abstract This article introduces a new problem of distributed knowledge graph, in IoT 5G setting. We developed an end‐to‐end solution for solving such problem by exploring the blockchain management and intelligent method for producing the better matching of the concepts and relations of the set of knowledge graphs. The concepts and the relations of the knowledge graphs are divided into several components, each of which contains similar concepts and relations. Instead of exploring the whole concepts and the relations of the knowledge graphs, only the representative of these components is compared during the matching process. The framework has outperformed state‐of‐the‐art knowledge graph matching algorithms using different scenarios as input in the experiments. In addition, to confirm the usability of our suggested framework, an in‐depth experimental analysis has been done; the results are very promising in both runtime and accuracy.
Part-wise Spatio-temporal Attention Driven CNN-based 3D Human Action Recognition

Recently, human activity recognition using skeleton data is increasing due to its ease of acquisition and finer shape details. Still, it suffers from a wide range of intra-class variation, inter-class similarity among the actions and view variation due to which extraction of discriminative spatial and temporal features is still a challenging problem. In this regard, we present a novel Residual Inception Attention Driven CNN (RIAC-Net) Network, which visualizes the dynamics of the action in a part-wise manner. The complete skeletonis partitioned into five key parts: Head to Spine, Left Leg, Right Leg, Left Hand, Right Hand. For each part, a Compact Action Skeleton Sequence (CASS) is defined. Part-wise skeleton-based motion dynamics highlights discriminative local features of the skeleton that helps to overcome the challenges of inter-class similarity and intra-class variation with improved recognition performance. The RIAC-Net architecture is inspired by the concept of inception-residual representation that unifies the Attention Driven Residues (ADR) with inception-based Spatio-Temporal Convolution Features (STCF) to learn efficient salient action features. An ablation study is also carried out to analyze the effect of ADR over simple residue-based action representation. The robustness of the proposed framework is evaluated by performing an extensive experiment on four challenging datasets: UT Kinect Action 3D, Florence 3D action, MSR Daily Action3D, and NTU RGB-D datasets, which consistently demonstrate the superiority of the proposed method over other state-of-the-art methods.
Dictionary-guided Scene Text Recognition

Language prior plays an important role in the way humans detect and recognize text in the wild. Current scene text recognition methods do use lexicons to improve recognition performance, but their naive approach of casting the output into a dictionary word based purely on the edit distance has many limitations. In this paper, we present a novel approach to incorporate a dictionary in both the training and inference stage of a scene text recognition system. We use the dictionary to generate a list of possible outcomes and find the one that is most compatible with the visual appearance of the text. The proposed method leads to a robust scene text recognition model, which is better at handling ambiguous cases encountered in the wild, and improves the overall performance of state-of-the-art scene text spotting frameworks. Our work suggests that incorporating language prior is a potential approach to advance scene text detection and recognition methods. Besides, we contribute VinText, a challenging scene text dataset for Vietnamese, where some characters are equivocal in the visual form due to accent symbols. This dataset will serve as a challenging benchmark for measuring the applicability and robustness of scene text detection and recognition algorithms. Code and dataset are available at https://github.com/VinAIResearch/dict-guided.
Isolated Sign Language Recognition with Multi-Scale Spatial-Temporal Graph Convolutional Networks

Isolated Sign Language Recognition (ISLR) fits nicely in the domain of problems that can be handled by graph-structured spatial-temporal algorithms. A recent multi-scale spatial-temporal graph convolution operator, MS-G3D, takes advantage of the semantic connectivity among non-neighbor nodes of the graph in a flexible temporal scale, which results in improved performance in classical Human Action Recognition datasets. In this work, we present a solution for ISLR using a skeleton graph that includes body and finger joints and makes use of this specific property of MS-G3D, which seems crucial to capture the internal relationship among semantically connected distant nodes in sign language dynamics. To complete the analysis, we compare the results with a 3D-CNN architecture, S3D, already used for SLR, and fuse it with MS-G3D. The performance achieved on the AUTSL dataset shows that MS-G3D alone stands out as a viable technique for ISLR. In fact, the improvement after fusing with a 3D-CNN approach, at least on this medium-scale dataset, appears marginal. The transfer learning capability of the trained models is also explored using pre-training with the larger WLASL dataset and post-training with the smaller LSE_UVIGO dataset. The classification performance based on the MS-G3D model over AUTSL does not benefit from pre-training with WLASL, but the performance on the more similarly acquired LSE_UVIGO dataset improves significantly from fine-tuning the MS-G3D AUTSL model.
Dynamic End Effector Tracking With an Omnidirectional Parallel Aerial Manipulator

To address the challenge of precise, dynamic and versatile aerial manipulation, we present an aerial manipulation platform consisting of a parallel 3-DOF manipulator mounted to an omnidirectional tilt-rotor aerial vehicle. The general modeling of a parallel manipulator on an omnidirectional floating base is presented, which motivates the optimization and detailed design of the aerial manipulator parameters and components. Inverse kinematic control of the manipulator is coupled to the omnidirectional base pose controller with a dynamic compensation term, going beyond common decoupled approaches. This presents a baseline for the control of redundant omnidirectional aerial manipulators. Experimental flights show the advantages of an active manipulator vs. a fixed arm for disturbance rejection and end effector tracking performance, as well as the practical limitations of the dynamic compensation term for fast end effector trajectories. The results motivate future studies for precise and dynamic aerial manipulation.
SAR-to-optical image translation based on improved CGAN

SAR images have the advantages of being less susceptible to clouds and light, while optical images conform to the human vision system. Both of them are widely applied in the field of scene classification, natural environment monitoring, disaster warning, etc. However, due to the speckle noise caused by the SAR imaging principle, it is difficult for people to distinguish the ground objects from complex background without professional knowledge. One commonly used solution is to exploit Generative Adversarial Networks (GAN) to translate SAR images to optical images which is able to clearly present ground objects with rich color information, i.e., SAR-to-optical image translation. Traditional GAN-based translation methods are apt to cause blurring of contour, disappearance of texture and inconsistency of color. To this end, we propose an improved conditional GAN (ICGAN) method. Compared with the basic CGAN model, the translation ability of our method is improved in the following three aspects. (1) Contour sharpness. We utilize the parallel branches to combine low-level and high-level features, and thus the image contour information is improved without the influence of noise. (2) Texture fine-grainedness. We discriminate the image using multi-scale receptive fields to enrich the local and global texture features of the image. (3) Color fidelity. We use the chromatic aberration loss which is based on Gaussian blur convolution to reduce the color gap between the generated image and the real optical image. Our method considers both the visual layer and the conceptual layer of the image to complete the SAR-to-optical image translation task. The model is able to preserve the contours and textures of the SAR image, while more closely approximates the colors of the ground truth. The experimental results show that the generated image not only has preferable results in visual effects and favorable evaluation metrics (subjective and objective), but also achieves outstanding classification accuracy, which proves the superiority of our method over the state-of-the-arts in the SAR-to-optical image translation task.
Hate Speech Detection in Roman Urdu

Hate speech is a specific type of controversial content that is widely legislated as a crime that must be identified and blocked. However, due to the sheer volume and velocity of the Twitter data stream, hate speech detection cannot be performed manually. To address this issue, several studies have been conducted for hate speech detection in European languages, whereas little attention has been paid to low-resource South Asian languages, making the social media vulnerable for millions of users. In particular, to the best of our knowledge, no study has been conducted for hate speech detection in Roman Urdu text, which is widely used in the sub-continent. In this study, we have scrapped more than 90,000 tweets and manually parsed them to identify 5,000 Roman Urdu tweets. Subsequently, we have employed an iterative approach to develop guidelines and used them for generating the Hate Speech Roman Urdu 2020 corpus. The tweets in the this corpus are classified at three levels: Neutral-Hostile, Simple-Complex, and Offensive-Hate speech. As another contribution, we have used five supervised learning techniques, including a deep learning technique, to evaluate and compare their effectiveness for hate speech detection. The results show that Logistic Regression outperformed all other techniques, including deep learning techniques for the two levels of classification, by achieved an F1 score of 0.906 for distinguishing between Neutral-Hostile tweets, and 0.756 for distinguishing between Offensive-Hate speech tweets.
BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning

Self-supervised learning in computer vision aims to pre-train an image encoder using a large amount of unlabeled images or (image, text) pairs. The pre-trained image encoder can then be used as a feature extractor to build downstream classifiers for many downstream tasks with a small amount of or no labeled training data. In this work, we propose BadEncoder, the first backdoor attack to self-supervised learning. In particular, our BadEncoder injects backdoors into a pre-trained image encoder such that the downstream classifiers built based on the backdoored image encoder for different downstream tasks simultaneously inherit the backdoor behavior. We formulate our BadEncoder as an optimization problem and we propose a gradient descent based method to solve it, which produces a backdoored image encoder from a clean one. Our extensive empirical evaluation results on multiple datasets show that our BadEncoder achieves high attack success rates while preserving the accuracy of the downstream classifiers. We also show the effectiveness of BadEncoder using two publicly available, real-world image encoders, i.e., Google’s image encoder pre-trained on ImageNet and OpenAI’s Contrastive Language-Image Pre-training (CLIP) image encoder pre-trained on 400 million (image, text) pairs collected from the Internet. Moreover, we consider defenses including Neural Cleanse and MNTD (empirical defenses) as well as PatchGuard (a provable defense). Our results show that these defenses are insufficient to defend against BadEncoder, highlighting the needs for new defenses against our BadEncoder. Our code is publicly available at: https://github.com/jjy1994/BadEncoder.
Table Tennis Stroke Recognition Using Two-Dimensional Human Pose Estimation

We introduce a novel method for collecting table tennis video data and perform stroke detection and classification. A diverse dataset containing video data of 11 basic strokes obtained from 14 professional table tennis players, summing up to a total of 22111 videos has been collected using the proposed setup. The temporal convolutional neural network model developed using 2D pose estimation performs multiclass classification of these 11 table tennis strokes with a validation accuracy of 99.37%. Moreover, the neural network generalizes well over the data of a player excluded from the training and validation dataset, classifying the fresh strokes with an overall best accuracy of 98.72%. Various model architectures using machine learning and deep learning based approaches have been trained for stroke recognition and their performances have been compared and benchmarked. Inferences such as performance monitoring and stroke comparison of the players using the model have been discussed. Therefore, we are contributing to the development of a computer vision based sports analytics system for the sport of table tennis that focuses on the previously unexploited aspect of the sport i.e., a player’s strokes, which is extremely insightful for performance improvement.