Reviewerly/exHarmony huggingface dataset:
Visual Analytics for RNN-Based Deep Reinforcement Learning

Deep reinforcement learning (DRL) targets to train an autonomous agent to interact with a pre-defined environment and strives to achieve specific goals through deep neural networks (DNN). Recurrent neural network (RNN) based DRL has demonstrated superior performance, as RNNs can effectively capture the temporal evolution of the environment and respond with proper agent actions. However, apart from the outstanding performance, little is known about how RNNs understand the environment internally and what has been memorized over time. Revealing these details is extremely important for deep learning experts to understand and improve DRLs, which in contrast, is also challenging due to the complicated data transformations inside these models. In this article, we propose Deep Reinforcement Learning Interactive Visual Explorer (DRLIVE), a visual analytics system to effectively explore, interpret, and diagnose RNN-based DRLs. Having focused on DRL agents trained for different Atari games, DRLIVE accomplishes three tasks: game episode exploration, RNN hidden/cell state examination, and interactive model perturbation. Using the system, one can flexibly explore a DRL agent through interactive visualizations, discover interpretable RNN cells by prioritizing RNN hidden/cell states with a set of metrics, and further diagnose the DRL model by interactively perturbing its inputs. Through concrete studies with multiple deep learning experts, we validated the efficacy of DRLIVE.
Three-Dimensional Autonomous Obstacle Avoidance Algorithm for UAV Based on Circular Arc Trajectory

This paper proposes an innovative and efficient three-dimensional (3D) autonomous obstacle algorithm for unmanned aerial vehicles (UAVs) which works by generating circular arc trajectories to avoid obstacles. Firstly, information on irregular obstacles is obtained by an onboard detection system; this information is then transformed into standard convex bodies, which are used to generate circular arc avoidance trajectories, and the obstacle avoidance problem is turned into a trajectory tracking strategy. Then, on the basis of the geometric relationship between a UAV and obstacle modeling, the working mechanism of the avoidance algorithm is developed. The rules of obstacle detection, avoidance direction, and the criterion of avoidance success are defined for different obstacle types. Finally, numerical simulations of different obstacle scenarios show that the proposed algorithm can avoid static and dynamic obstacles effectively and can implement obstacle avoidance missions for UAVs well.
Coverless Steganography Based on Motion Analysis of Video

With the rapid development of interactive multimedia services and camera sensor networks, the number of network videos is exploding, which has formed a natural carrier library for steganography. In this study, a coverless steganography scheme based on motion analysis of video is proposed. For every video in the database, the robust histograms of oriented optical flow (RHOOF) are obtained, and the index database is constructed. The hidden information bits are mapped to the hash sequences of RHOOF, and the corresponding indexes are sent by the sender. At the receiver, through calculating hash sequences of RHOOF from the cover video, the secret information can be extracted successfully. During the whole process, the cover video remains original without any modification and has a strong ability to resist steganalysis. The capacity is investigated and shows good improvement. The robustness performance is prominent against most attacks such as pepper and salt noise, speckle noise, MPEG-4 compression, and motion JPEG 2000 compression. Compared with the existing coverless information hiding schemes based on images, the proposed method not only obtains a good trade-off between hiding information capacity and robustness but also can achieve higher hiding success rate and lower transmission data load, which shows good practicability and feasibility.
Political Ecologies of Participation

This paper presents an unexpected story about the outcomes of a civic project. CSCW and HCI scholarship has argued for a long-term perspective to assess civic projects and to understand how local communities appropriate - or maybe disregard - the material outcomes of these types of interventions. Nevertheless, it is still unclear how to interpret the outcomes of such projects and how to make sense of their social impact on the wider contexts they are bound to. This article draws on the notion of "political ecologies of participation" to illustrate: i) how outcomes of socially engaged projects circulate through communities and can be appropriated independently of the research inquiries they stem from; and ii) how, through such processes, impact is reconfigured as issues are added to shared concerns. The paper sets out by analyzing the entanglement of actors, meanings (e.g., values, narratives, opinions) and forms of participation that were configured throughout the transformation observed. Attention is then drawn to their political qualities, including their inherent openness and their capacity to produce change locally. The paper introduces four analytical sensitivities illustrating how thinking with political ecologies of participation can help CSCW research focus on the longer processes whereby impact can be configured.
Adaptive Laplacian Support Vector Machine for Semi-supervised Learning

Abstract Laplacian support vector machine (LapSVM) is an extremely popular classification method and relies on a small number of labels and a Laplacian regularization to complete the training of the support vector machine (SVM). However, the training of SVM model and Laplacian matrix construction are usually two independent process. Therefore, In this paper, we propose a new adaptive LapSVM method to realize semi-supervised learning with a primal solution. Specifically, the hinge loss of unlabelled data is considered to maximize the distance between unlabelled samples from different classes and the process of dealing with labelled data are similar to other LapSVM methods. Besides, the proposed method embeds the Laplacian matrix acquisition into the SVM training process to improve the effectiveness of Laplacian matrix and the accuracy of new SVM model. Moreover, a novel optimization algorithm considering primal solver is proposed to our adaptive LapSVM model. Experimental results showed that our method outperformed all comparison methods in terms of different evaluation metrics on both real datasets and synthetic datasets.
Adaptive Spatio-Temporal Graph Enhanced Vision-Language Representation for Video QA

Vision-language research has become very popular, which focuses on understanding of visual contents, language semantics and relationships between them. Video question answering (Video QA) is one of the typical tasks. Recently, several BERT style pre-training methods have been proposed and shown effectiveness on various vision-language tasks. In this work, we leverage the successful vision-language transformer structure to solve the Video QA problem. However, we do not pre-train it with any video data, because video pre-training requires massive computing resources and is hard to perform with only a few GPUs. Instead, our work aims to leverage image-language pre-training to help with video-language modeling, by sharing a common module design. We further introduce an adaptive spatio-temporal graph to enhance the vision-language representation learning. That is, we adaptively refine the spatio-temporal tubes of salient objects according to their spatio-temporal relations learned through a hierarchical graph convolution process. Finally, we can obtain a number of fine-grained tube-level video object representations, as the visual inputs of the vision-language transformer module. Experiments on three widely used Video QA datasets show that our model achieves the new state-of-the-art results.
Virtual reality game level layout design for real environment constraints

This paper presents an optimization-based approach for designing virtual reality game level layouts, based on the layout of a real environment. Our method starts by asking the user to define the shape of the real environment and the obstacles (e.g., furniture) located in it. Then, by representing a game level as an assembly of chunks and defining the game level layout design decisions in cost terms (mapping, fitting, variations, and accessibility) in a total cost function, our system automatically synthesizes a game level layout that fulfills the real environment layout and its constraints as well as the user-defined design decisions. To evaluate the proposed method, a user study was conducted. The results indicated that the proposed method: (1) enhanced the levels of presence; (2) enhanced the levels of involvement of participants in the virtual environment; and (3) reduced the fear of collision with the real environment and its constraints. Limitations and future research directions are also discussed.
Digitalisation of Teaching and Learning as a Tool for Increasing Students’ Satisfaction and Educational Efficiency: Using Smart Platforms in EFL

This manuscript aims to present possibilities for developing mobile and smart platforms and systems in teaching and learning the English language for engineering professionals in different engineering study programs. Foreign language teaching and learning processes are based on traditional methods, while in engineering and technical sciences, teaching and learning processes include different digital platforms. Therefore, the following hypotheses were stated. (H1) It is possible to develop a software solution for mobile platforms that can have a higher level of interactivity, and it may lead to better learning outcomes, especially in the field of adopting engineering vocabulary. (H2) Implementation of the developed solution increases motivation for learning and leads to a higher level of satisfaction with the learning process as a part of the quality of life. (H3) Students who have digital and mobile platforms in the learning process could have higher achievement values. This manuscript presents software application development and its implementation in teaching English as a foreign language for engineering and technical study programs on the bachelor level. Initial results in implementation and satisfaction of end users point to the justification of implementing such solutions.
Double anchor embedding for accurate multi-person 2D pose estimation

Multi-person pose estimation is an important field in computer vision. Due to the lower time complexity, the bottom-up approaches have recently received more attention in multi-person 2D pose estimation, however, they are more sensitive to challenges in real-world scenarios. In this paper, we propose a multi-person pose estimation algorithm based on the Double Anchor Embedding (DAE), which shows that bottom-up algorithms are still competitive in precision. Firstly, for reducing the modeling difficulty of the detection task we divide the human joints into upper and lower half groups which are internally continuous and highly correlated. Accordingly, a novel joint affinity cue, called Double Anchor Embedding is designed, which can help the network effectively extract the information of both local contexts and global contexts, so that can better cope with occluded scenes and complex postures. Secondly, the parallel greedy joint inference algorithm is proposed to alleviate the mismatching problem of distant joints in the post-processing stage, which can also accelerate the matching process to some extent. Extensive experiments on two challenging datasets demonstrate the effectiveness and potential of our proposed framework, which is comparable to the current state-of-the-art methods.
Proactive auto-scaling for cloud environments using temporal convolutional neural networks

Auto-scaling systems can dynamically scale the required resources for cloud-based services at runtime. This is an effective mechanism, enabling services to adapt to environmental changes. These systems establish the foundation for achieving elasticity in the modern cloud computing paradigm. Given the dynamic and uncertain nature of the shared cloud infrastructure, cloud auto-scaling systems are one of the most complex and sophisticated created artifacts, aiming to achieve self-aware, self-adaptive, and dependable runtime scaling. To find an effective solution to this problem, an accurate prediction of the required amount of workload as well as the system metrics for future time periods are needed. Various solutions have already been proposed to tackle this problem. Many solutions make use of machine learning, statistical, and ensemble methods. In this paper, we view the auto-scaling problem as a sequence model and apply the convolutional neural networks to predict the future workload of cloud services. Also, by using neural networks, we obtain a mapping between the predicted workload as well as the real-time and future amounts of the required resources. We have also proposed a decision-making mechanism that takes into account different and sometimes conflicting user criteria resulting in the best-compromised decision. To this aim, we have used TOPSIS as a multi-criteria decision-making method for the decision-making component. In the evaluation section, we have examined the amount of prediction error, the amount of service level agreement violations, as well as the amount of resources' under-utilization. Evaluations demonstrate that the proposed approach for predicting the workload shows a 4 percent improvement over the existing approaches.
SmartCAMPP - Smartphone-based continuous authentication leveraging motion sensors with privacy preservation

Continuous Authentication (CA) approaches are attracting attention due to the explosion of available sensors from IoT devices such as smartphones. However, a critical privacy concern arises when CA data is outsourced. Data from motion sensors may reveal users’ private issues. Despite the need for CA in smartphones, no previous work has explored how to tackle this matter leveraging motion sensors in a privacy-preserving way. In this work, a mechanism dubbed SmartCAMPP is proposed to achieve CA based on gyroscope and accelerometer data. Format-preserving encryption techniques are applied to privately outsource them. Our results show the suitability of the proposed scheme, featuring 76.85% of accuracy while taking 5.12 ms. of computation for authenticating each user. Interestingly, the use of cryptography does not lead to a significant impact as compared to a non-privacy-preserving mechanism.
An Empirical Analysis of Pool Hopping Behavior in the Bitcoin Blockchain

We provide an empirical analysis of pool hopping behavior among 15 mining pools throughout Bitcoin's history.Mining pools have emerged as major players to ensure that the Bitcoin system stays secure, valid, and stable.Individual miners join mining pools to benefit from a more predictable income.Many questions remain open regarding how mining pools have evolved throughout Bitcoin's history and when and why miners join or leave mining pools.We propose a heuristic algorithm to extract the payout flow from mining pools and detect the pools' migration of miners.Our results showed that payout schemes and pool fees influence miners' decisions to join, change, or exit from a mining pool, thus affecting the dynamics of mining pool market shares.Our analysis provides evidence that mining activity becomes an industry as miners' decisions follow classical economic rationale.
Voicemoji: Emoji Entry Using Voice for Visually Impaired People

Keyboard-based emoji entry can be challenging for people with visual impairments: users have to sequentially navigate emoji lists using screen readers to find their desired emojis, which is a slow and tedious process. In this work, we explore the design and benefits of emoji entry with speech input, a popular text entry method among people with visual impairments. After conducting interviews to understand blind or low vision (BLV) users’ current emoji input experiences, we developed Voicemoji, which (1) outputs relevant emojis in response to voice commands, and (2) provides context-sensitive emoji suggestions through speech output. We also conducted a multi-stage evaluation study with six BLV participants from the United States and six BLV participants from China, finding that Voicemoji significantly reduced entry time by 91.2% and was preferred by all participants over the Apple iOS keyboard. Based on our findings, we present Voicemoji as a feasible solution for voice-based emoji entry.
Auth+Track: Enabling Authentication Free Interaction on Smartphone by Continuous User Tracking

We propose Auth+Track, a novel authentication model that aims to reduce redundant authentication in everyday smartphone usage. By sparse authentication and continuous tracking of the user’s status, Auth+Track eliminates the “gap” authentication between fragmented sessions and enables “Authentication Free when User is Around”. To instantiate the Auth+Track model, we present PanoTrack, a prototype that integrates body and near field hand information for user tracking. We install a fisheye camera on the top of the phone to achieve a panoramic vision that can capture both user’s body and on-screen hands. Based on the captured video stream, we develop an algorithm to extract 1) features for user tracking, including body keypoints and their temporal and spatial association, near field hand status, and 2) features for user identity assignment. The results of our user studies validate the feasibility of PanoTrack and demonstrate that Auth+Track not only improves the authentication efficiency but also enhances user experiences with better usability.
Temporal Attention-Augmented Graph Convolutional Network for Efficient Skeleton-Based Human Action Recognition

Graph convolutional networks (GCNs) have been very successful in modeling non-Euclidean data structures, like sequences of body skeletons forming actions modeled as spatiotemporal graphs. Most GCN-based action recognition methods use deep feed-forward networks with high computational complexity to process all skeletons in an action. This leads to a high number of floating point operations (ranging from 16G to 100G FLOPs) to process a single sample, making their adoption in restricted computation application scenarios infeasible. In this paper, we propose a temporal attention module (TAM) for increasing the efficiency in skeleton-based action recognition by selecting the most informative skeletons of an action at the early layers of the network. We incorporate the TAM in a lightweight GCN topology to further reduce the overall number of computations. Experimental results on two benchmark datasets show that the proposed method outperforms with a large margin the baseline GCN-based method while having ×2.9 less number of computations. Moreover, it performs on par with the state-of-the-art with up to ×9.6 less number of computations.
Image Segmentation Technology and Its Application in Digital Image Processing

In recent years, the level of science and technology in China has been significantly improved. Digital image processing technology is the product of the times with the rapid development of science and technology, and has been applied in many fields. Digital image processing technology includes many practical technologies, and image segmentation technology is one of them. At present, the widely used pattern recognition technology is realized by image segmentation technology. Based on this, this paper mainly studies the application of image segmentation technology in digital image processing. This paper first introduces the methods of image segmentation, including threshold segmentation, clustering segmentation and edge detection segmentation. Secondly, according to a need to extract a road image for image segmentation, and achieved very good results. Finally, this paper mainly analyzes the application of image segmentation in digital image processing. It is found that image segmentation has good application in automatic license plate recognition, biomedical engineering, remote sensing engineering and fire prevention and detection.
Design and Implementation of an Efficient Elliptic Curve Digital Signature Algorithm (ECDSA)

Digital signatures are increasingly used today. It replaces wet signature with the development of technology. Elliptic curve digital signature algorithm (ECDSA) is used in many applications thanks to its security and efficiency. However, some mathematical operations such as inversion operation in modulation slow down the speed of this algorithm. In this study, we propose a more efficient and secure ECDSA. In the proposed method, the inversion operation in modulation of signature generation and signature verification phases is removed. Thus, the efficiency and speed of the ECDSA have been increased without reducing its security. The proposed method is implemented in Python programming language using P-521 elliptic curve and SHA-512 algorithm.
Asset Management in Machine Learning: A Survey

Machine Learning (ML) techniques are becoming essential components of many software systems today, causing an increasing need to adapt traditional software engineering practices and tools to the development of ML-based software systems. This need is especially pronounced due to the challenges associated with the large-scale development and deployment of ML systems. Among the most commonly reported challenges during the development, production, and operation of ML-based systems are experiment management, dependency management, monitoring, and logging of ML assets. In recent years, we have seen several efforts to address these challenges as witnessed by an increasing number of tools for tracking and managing ML experiments and their assets. To facilitate research and practice on engineering intelligent systems, it is essential to understand the nature of the current tool support for managing ML assets. What kind of support is provided? What asset types are tracked? What operations are offered to users for managing those assets? We discuss and position ML asset management as an important discipline that provides methods and tools for ML assets as structures and the ML development activities as their operations. We present a feature-based survey of 17 tools with ML asset management support identified in a systematic search. We overview these tools' features for managing the different types of assets used for engineering ML-based systems and performing experiments. We found that most of the asset management support depends on traditional version control systems, while only a few tools support an asset granularity level that differentiates between important ML assets, such as datasets and models.
Context Matters: Self-Attention for Sign Language Recognition

This paper proposes an attentional network for the task of Continuous Sign Language Recognition. The proposed approach exploits co-independent streams of data to model the sign language modalities. These different channels of information can share a complex temporal structure between each other. For that reason, we apply attention to synchronize and help capture entangled dependencies between the different sign language components. Even though Sign Language is multi-channel, handshapes represent the central entities in sign interpretation. Seeing handshapes in their correct context defines the meaning of a sign. Taking that into account, we utilize the attention mechanism to efficiently aggregate the hand features with their appropriate spatio-temporal context for better sign recognition. We found that by doing so the model is able to identify the essential Sign Language components that revolve around the dominant hand and the face areas. We test our model on the benchmark dataset RWTH-PHOENIX-Weather 2014, yielding competitive results.