Reviewerly/exHarmony huggingface dataset:
Exploring Multi‐dimensional Data via Subset Embedding

Abstract Multi‐dimensional data exploration is a classic research topic in visualization. Most existing approaches are designed for identifying record patterns in dimensional space or subspace. In this paper, we propose a visual analytics approach to exploring subset patterns. The core of the approach is a subset embedding network (SEN) that represents a group of subsets as uniformly‐formatted embeddings. We implement the SEN as multiple subnets with separate loss functions. The design enables to handle arbitrary subsets and capture the similarity of subsets on single features, thus achieving accurate pattern exploration, which in most cases is searching for subsets having similar values on few features. Moreover, each subnet is a fully‐connected neural network with one hidden layer. The simple structure brings high training efficiency. We integrate the SEN into a visualization system that achieves a 3‐step workflow. Specifically, analysts (1) partition the given dataset into subsets, (2) select portions in a projected latent space created using the SEN, and (3) determine the existence of patterns within selected subsets. Generally, the system combines visualizations, interactions, automatic methods, and quantitative measures to balance the exploration flexibility and operation efficiency, and improve the interpretability and faithfulness of the identified patterns. Case studies and quantitative experiments on multiple open datasets demonstrate the general applicability and effectiveness of our approach.
Secure and Usable QR Codes for Healthcare Systems: The Case of Covid-19 Pandemic

Recently, especially during the COVID-19 pandemic, there is a growing interest in employing secure technology in health care systems, enhancing workflow, reducing costs, and developing the flexibility of providing healthcare services to users. QR codes are accessible, free, easy to use, and can be scanned through several free apps on smartphones. QR codes can help healthcare systems as an effective alternative to using paper-based documents for a patient identity system. Sensitive information can be stored in the QR codes of patients and medical staff as offline data or stored in a particular URL. Besides, QR codes can hold prescriptions and electronic permits used during the lockdown period. As there is no standard structure or authentication phase in QR code generation, such codes are vulnerable to fake content. This study's main objective is to protect healthcare information embedded inside QR codes by employing cryptographic mechanisms. Several experiments were conducted to determine how different features and users deal with barcode scanning. We introduce an authentication and integrity phase during QR code generation. We integrated digital signature, encryption, and access control techniques into a secure, real-time barcode generator and scanner implementation to protect QR code content against unauthorized users. The results demonstrate that the proposed system enables generating secure and usable QR codes for healthcare information protection among several scenarios.
UniKG: A Unified Interoperable Knowledge Graph Database System

Knowledge graph currently has two main data models: RDF graph and property graph. The query language on RDF graph is SPARQL, while the query language on property graph is mainly Cypher. Different data models and query languages hinder the wider application of knowledge graphs. In this demonstration, we propose a unified interoperable knowledge graph database system, UniKG. (1) Based on the relational model, a unified storage scheme is utilized to efficiently store RDF graphs and property graphs, and support the query requirements of knowledge graphs. (2) Using the characteristicset-based method, the storage problem of untyped entities is addressed in UniKG. (3) UniKG realizes the interoperability of SPARQL and Cypher, and enables them to interchangeably operate on the same knowledge graph. (4) With a unified Web interface, users are allowed to query with two different languages over the same knowledge graph and visualize query results and explanations.
Development of complete image processing system including image filtering, image compression &amp; image security

Computer vision is nowadays is one of the promising & evolving areas in information technology. Image processing is increasingly used in several applications such as automotive, medical or aerospace. Every computer vision application involves capturing several images with the help of camera. Image acquisition, image transmission & image security are the three important aspects of image processing system. This paper includes development of complete image processing system including improved median filtering with better PSNR & operating frequency, Image compression module with better PSNR & compression ratio, Image Encryption module using Advanced Encryption Standard.
Multi-Objective Heuristic Decision Making and Benchmarking for Mobile Applications in English Language Learning

This research proposes to evaluate and analyze the decision matrix for learner's English mobile applications (EMAs) based on multi-objective heuristic decision making with a view to listening, speaking, reading, and writing. Because of the number of criteria, the significance of parameters, and variance in results, EMAs are difficult. Decision making has built on the combination of listening, speaking, reading, and writing and EMA evaluation criteria for students. The requirements are adapted from a framework of pre-school education. Six alternatives and 17 skills as a requirement are included in decision-making results. The six EMA are then assessed, with six English learning experts distributing a review form. The application subsequently is evaluated using the best-worst method and preference-order technique (TOPSIS) using multi-objective heuristic decision making methods. The best-worst method is used to measure requirements, whereas TOPSIS is used to test and assess the applications. In two cases, namely person and group, TOPSIS is used. Internal and external aggregations are used throughout the group context. In effect, the aim of evaluating the proposed study and comparing it to six relative studies with scenarios and benchmarking checklists is to develop an objectives validation framework for e-apps.
Enrichment of Oesophageal Speech: Voice Conversion with Duration–Matched Synthetic Speech as Target

Pathological speech such as Oesophageal Speech (OS) is difficult to understand due to the presence of undesired artefacts and lack of normal healthy speech characteristics. Modern speech technologies and machine learning enable us to transform pathological speech to improve intelligibility and quality. We have used a neural network based voice conversion method with the aim of improving the intelligibility and reducing the listening effort (LE) of four OS speakers of varying speaking proficiency. The novelty of this method is the use of synthetic speech matched in duration with the source OS as the target, instead of parallel aligned healthy speech. We evaluated the converted samples from this system using a collection of Automatic Speech Recognition systems (ASR), an objective intelligibility metric (STOI) and a subjective test. ASR evaluation shows that the proposed system had significantly better word recognition accuracy compared to unprocessed OS, and baseline systems which used aligned healthy speech as the target. There was an improvement of at least 15% on STOI scores indicating a higher intelligibility for the proposed system compared to unprocessed OS, and a higher target similarity in the proposed system compared to baseline systems. The subjective test reveals a significant preference for the proposed system compared to unprocessed OS for all OS speakers, except one who was the least proficient OS speaker in the data set.
Adaptive Visualisations Using Spatiotemporal and Heuristic Models to Support Piano Learning

Learning the piano is hard and many approaches including piano-roll visualisations have been explored in order to support novices and seasoned learners in this process. However, existing piano roll prototypes have not considered the spatiotemporal component (user’s ability to press on a moving target) when generating these visualisations and user modelling. In this PhD, we are going to look into two different approaches: (i) exploring whether existing techniques in single-target spatiotemporal modelling can be adapted to a multi-target scenario such as when learners use several fingers to press multiple moving targets when playing the piano, and (ii) exploring heuristics defined by experts marking various difficult parts of songs, and deciding on specific interventions needed for these marked parts. Using models and input from the experts we will design and build an adaptive piano roll training system. We will evaluate and compare these models in various user studies involving users trying to play piano pieces and develop their improvisation skills. We intend to uncover whether these adaptive visualisations will be helpful in the overall training of piano learners. Additionally, these models and adaptive visualisations will allow us to discover affordances that can potentially improve piano learning in general.
Model-Aware Gesture-to-Gesture Translation

Hand gesture-to-gesture translation is a significant and interesting problem, which serves as a key role in many applications, such as sign language production. This task involves fine-grained structure understanding of the mapping between the source and target gestures. Current works follow a data-driven paradigm based on sparse 2D joint representation. However, given the insufficient representation capability of 2D joints, this paradigm easily leads to blurry generation results with incorrect structure. In this paper, we propose a novel model-aware gesture-to-gesture translation framework, which introduces hand prior with hand meshes as the intermediate representation. To take full advantage of the structured hand model, we first build a dense topology map aligning the image plane with the encoded embedding of the visible hand mesh. Then, a transformation flow is calculated based on the correspondence of the source and target topology map. During the generation stage, we inject the topology information into generation streams by modulating the activations in a spatially-adaptive manner. Further, we incorporate the source local characteristic to enhance the translated gesture image according to the transformation flow. Extensive experiments on two benchmark datasets have demonstrated that our method achieves new state-of-the-art performance.
Exploring Affordances of Surface Gestures on Textile User Interfaces

This pictorial explores the design space for communicating surface gestures to users of textile interfaces by experimenting with the interfaces' physical design and affordances. First, we created a collection of functional and non-functional textile samples. Their development was based on three aspects: design, fabrication, and sensing. The design aspect covered different visual (shape, color) and haptic (details, textures) designs, fabrication explored three textile-specific fabrication methods, and electronic sensing offered options for adding touch-sensing capabilities. Second, we reflected on created samples and their characteristics contrasting different designs and speculating on why some work better than others. Our main findings and insights are presented in five clusters: ergonomics, visual affordances, perception of textures, the direction of movement, and the economic usage of design elements. This intermediate-level knowledge can provide a starting point for each professional or novice designer to take inspiration from, when creating their own textile user interfaces.
RMS-SE-UNet: A Segmentation Method for Tumors in Breast Ultrasound Images

It is a significant challenge to obtain accurate boundary of tumors due to much speckle noises. In this paper, we proposed some meaningful modules to address the problem. Firstly, a large amount of semantic information is needed to determine the boundary on account of fuzziness around the edge of breast tumor, we proposed residual multi-scale(RMS) block to collect larger receptive filed. Secondly, we redesigned the skip connections and combined Squeeze-and-Excitation(SE) block to incorporate information from different layers. Finally we introduced deep supervision and hybrid loss function to accelerate the convergence of network. Dice similarity coefficient and intersection over union(IoU) were used to evaluate segmentation results which were 94.69 % and 90.01 % respectively on test set. It is shown that our method is effective in this kind of problem.
Multi-UAV cooperative Route planning based on decision variables and improved genetic algorithm

Abstract In order to reduce the threat cost of multi-UAV coordination, improve the effective flight time and the mission success rate, a path planning model was designed for the effectiveness and real-time performance of multi-UAV track coordination. In this paper, based on the path planning model under the condition of non-interference, the model of minimum cost flight path of multi-UAV based on time synergy is given, and a time-sequence-space coordination model is proposed to meet the requirements of electronic jamming, mission confusion and radar illumination. The decision variables of range and threat are introduced into the collaborative function of track cost, and by adjusting the cost functions and collaborative variables in real time, the path planning model was optimized for unmanned support aircraft in coordination with multiple unmanned aircraft mission, at the same time, the genetic algorithm is improved by combining the adaptive crossover rate and mutation rate, and the optimization of track collaborative solution process is realized. Finally, the effectiveness of track coordination model of multi-UAV based on decision variables and improved genetic algorithm is verified by semi-physical simulation.
Distributed State Estimation for Stochastic Linear Hybrid Systems With Finite-Time Fusion

Building on the classical interacting multiple model algorithm, this article develops a novel distributed approach for solving the problem of distributed state estimation for stochastic linear hybrid systems. In our distributed framework, a network of sensors is employed and each of them measures only a portion of the system outputs, thus, the system may not necessarily be observable for any individual sensor. To tackle with this, we develop an effective data fusion scheme that enables the sensor network to collectively measure the output of the hybrid system, by leveraging the communication between neighboring sensors. Consequently, each sensor only needs to process a relatively small set of data and is able to locally and identically observe the states (both the continuous states and discrete modes) of stochastic linear hybrid systems. Stability of the proposed distributed algorithm is proved theoretically, in the sense that the covariance of estimation error is lower and upper bounded. Finally, the performance of the proposed algorithm is demonstrated via illustrative simulations on a maneuvering aircraft tracking problem.
Gesture-Controlled Robotic Arm Utilizing OpenCV

In this paper, a low-cost, 3D printed robotic arm that uses a system of gesture recognition controlled via computer vision is presented. The developed approach of Human-Computer Interaction (HCI) employs a single commonly available USB2.0 High-Definition camera to capture hand movements and gestures, implementing the OpenCV library. This enables the translation of those movements and gestures into commands that can be transferred using internet protocols to a Raspberry Pi; then interfaces with the robotic arm. Using computer vision as a method of HCI reduces components and overall cost. The Raspberry Pi uses individual commands to control the robotic arm's servo motors while monitoring multiple pressure sensors and limit switches to ensure the robotic arm operates within a certain limit of load. This prevents the arm from overextending itself while protecting the integrity of items within its grasp.
IoT Based Smart Wheelchair for Elderly Healthcare Monitoring

Objective: Development of a prototype wheelchair system that uses internet of thing (IoT) biophysical sensors for AI healthcare monitoring and assistive technology to support the independence of elderly patients. Algorithms are applied to analyse sensor data to provide feedback in real-time to the user and clinicians on risk factors. Methods: The system incorporated multiple sources of vital signs including body temperature, blood pressure, heart rate, oxygen saturation with AI in an integrated user interface and an autonomously navigated wheelchair. Results: A prototype of the assistive powered wheelchair was developed. Biophysical sensors were embedded in the wheelchair to collect patients' vital signs in real-time over ten seconds intervals and wirelessly uploaded to a cloud every forty seconds. A user interface was developed to record, visualise, and analyse patients' data for doctors and caregivers. Conclusion: The smart wheelchair will help patients drive autonomously within a predefined area. Vital sign signals from patients can be collected and analysed remotely. Further improvements can include use of different biophysical sensors, including for monitoring of falls and posture, and further development of algorithms to allow better management of patients' chronic condition.
Emotion-sensitive deep dyna-Q learning for task-completion dialogue policy learning

In recent years, task-oriented dialogue systems have received extensive attention from academia and industry. Training a dialogue agent through reinforcement learning is often costly because it requires many interactions with real users. Although the Deep Dyna-Q (DDQ) framework uses simulation experience to alleviate the cost of direct reinforcement learning, it still suffers from challenges such as delayed rewards and policy degradation. This paper proposes an Emotion-Sensitive Deep Dyna-Q (ES-DDQ) model which: (1) presents an emotional world model that considers emotion-related cues to improve the ability of the traditional DDQ framework to model and simulate users, and (2) designs two kinds of emotion-related immediate rewards to mitigate the delayed reward problem. Experimental results show that our proposed approach effectively simulates users' behaviors and is superior to the state-of-the-art benchmarks.
An Efficient Text Detection Model for Street Signs

Text detection in natural scenes is a current research hotspot. The Efficient and Accurate Scene Text (EAST) detector model has fast detection speed and good performance but is ineffective in detecting long text regions owing to its small receptive field. In this study, we built upon the EAST model by improving the bounding box’s shrinking algorithm to make the model more accurate in predicting short edges of text regions; altering the loss function from balanced cross-entropy to Focal loss; improving the model’s learning ability on hard, positive examples; and adding a feature enhancement module (FEM) to increase the receptive field of the EAST model and enhance its detection ability for long text regions. The improved EAST model achieved better detection results on both the ICDAR2015 dataset and the Street Sign Text Detection (SSTD) dataset proposed in this paper. The precision and F1 scores of the model also demonstrated advantages over other models on the ICDAR2015 dataset. A comparison of the text detection effects between the improved EAST model and the EAST model showed that the proposed FEM was more effective in increasing the EAST detector’s receptive field, which indicates that it can improve the detection of long text regions.
Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size

In the last two decades, information entropy measures have been relevantly applied in fuzzy clustering problems in order to regularize solutions by avoiding the formation of partitions with excessively overlapping clusters. Following this idea, relative entropy or divergence measures have been similarly applied, particularly to enable that kind of entropy-based regularization to also take into account, as well as interact with, cluster size variables. Particularly, since Rényi divergence generalizes several other divergence measures, its application in fuzzy clustering seems promising for devising more general and potentially more effective methods. However, previous works making use of either Rényi entropy or divergence in fuzzy clustering, respectively, have not considered cluster sizes (thus applying regularization in terms of entropy, not divergence) or employed divergence without a regularization purpose. Then, the main contribution of this work is the introduction of a new regularization term based on Rényi relative entropy between membership degrees and observation ratios per cluster to penalize overlapping solutions in fuzzy clustering analysis. Specifically, such Rényi divergence-based term is added to the variance-based Fuzzy C-means objective function when allowing cluster sizes. This then leads to the development of two new fuzzy clustering methods exhibiting Rényi divergence-based regularization, the second one extending the first by considering a Gaussian kernel metric instead of the Euclidean distance. Iterative expressions for these methods are derived through the explicit application of Lagrange multipliers. An interesting feature of these expressions is that the proposed methods seem to take advantage of a greater amount of information in the updating steps for membership degrees and observations ratios per cluster. Finally, an extensive computational study is presented showing the feasibility and comparatively good performance of the proposed methods.
Adaptive Binary-Ternary Quantization

Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous vehicles. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary quantization provides a more flexible model and outperforms binary quantization in terms of accuracy, however doubles the memory footprint and increases the computational cost. Contrary to these approaches, mixed quantized models allow a trade-off between accuracy and memory footprint. In such models, quantization depth is often chosen manually, or is tuned using a separate optimization routine. The latter requires training a quantized network multiple times. Here, we propose an adaptive combination of binary and ternary quantization, namely Smart Quantization (SQ), in which the quantization depth is modified directly via a regularization function, so that the model is trained only once. Our experimental results show that the proposed method adapts quantization depth successfully while keeping the model accuracy high on MNIST and CIFAR10 benchmarks.
Self-Supervised Deep Metric Learning for Pointsets

Deep metric learning is a supervised learning paradigm to construct a meaningful vector space to represent complex objects. A successful application of deep metric learning to pointsets means that we can avoid expensive retrieval operations on objects such as documents and can significantly facilitate many machine learning and data mining tasks involving pointsets. We propose a self-supervised deep metric learning solution for pointsets. The novelty of our proposed solution lies in a self-supervision mechanism that makes use of a distribution distance for set ranking called the Earth's Mover Distance (EMD) to generate pseudo labels. Our experimental studies on four documents datasets show that our proposed solutions outperform baselines and state-of-the-art approaches on unsupervised deep metric learning in most settings.