Reviewerly/exHarmony huggingface dataset:
A Multi-Object Grasp Technique for Placement of Objects in Virtual Reality

Some daily tasks involve grasping multiple objects in one hand and releasing them in a determined order, for example laying out a surgical table or distributing items on shelves. For training these tasks in Virtual Reality (VR), there is no technique for allowing users to grasp multiple objects in one hand in a realistic way, and it is not known if such a technique would benefit user experience. Here, we design a multi-object grasp technique that enables users to grasp multiple objects in one hand and release them in a controlled way. We tested an object placement task under three conditions: real life, VR with single-object grasp and VR with multi-object grasp. Task completion time, distance travelled by the hands and subjective experience were measured in three scenarios: sitting in front of a desktop table, standing up in front of shelves and a room-size scenario where walking was required. Results show that the performance in a real environment is better than in Virtual Reality, both for single-object and multi-object grasping. The single-object technique performs better than the multi-object, except for the room scenario, where multi-object leads to less distance travelled and reported physical demand. For use cases where the distances are small (i.e., desktop scenario), single-object grasp is simpler and easier to understand. For larger scenarios, the multi-object grasp technique represents a good option that can be considered by other application designers.
OMG: Observe Multiple Granularities for Natural Language-Based Vehicle Retrieval

Retrieving tracked-vehicles by natural language descriptions plays a critical role in smart city construction. It aims to find the best match for the given texts from a set of tracked vehicles in surveillance videos. Existing works generally solve it by a dual-stream framework, which consists of a text encoder, a visual encoder and a cross-modal loss function. Although some progress has been made, they failed to fully exploit the information at various levels of granularity. To tackle this issue, we propose a novel framework for the natural language-based vehicle retrieval task, OMG, which Observes Multiple Granularities with respect to visual representation, textual representation and objective functions. For the visual representation, target features, context features and motion features are encoded separately. For the textual representation, one global embedding, three local embeddings and a color-type prompt embedding are extracted to represent various granularities of semantic features. Finally, the overall framework is optimized by a cross-modal multi-granularity contrastive loss function. Experiments demonstrate the effectiveness of our method. Our OMG significantly outperforms all previous methods and ranks the 9th on the 6th AI City Challenge Track2. The codes are available at https://github.com/dyhBUPT/OMG.
Handwritten signatures verification based on arm and hand muscles synergy

Intra-personal variability, which measures the difference between people's signatures, may be affected by various signing challenges. Considering the variety of physical conditions, it is almost impossible for persons to write their exact handwritten signature in the same way in several attempts. Also, the authentication system requires less complexity to respond quickly to real-time applications. This study attempts to confirm handwritten signatures by using hand muscle synergy as a biometric characteristic. To design the signature verification system, surface electromyography (EMG) signals from eight (arm and hand) muscles of the volunteers were recorded by surface EMG pads during the signing. Muscle synergy was extracted from EMG signals after preprocessing using the non-negative matrix factorization (NMF) method. Genuine and forgery data are then classified by the K-means classifier. The system achieves an equal error rate (EER) of 2.75 to identify the extracted data related to the genuine and forged signatures.
If I Share with you my Perspective, Would you Share your Data with me?

Real-time 3D reconstruction using multiple RGBD cameras and their online transmission facilitates the adoption of mixed reality telepresence. However, such a system can only cover a limited volume, and increasing the number of RGBD cameras is unfeasible due to setup complexity and space constraints. To address this issue, we present the concept of Dynamic 3D View Sharing, which complements the views of a 3D reconstruction system by the dynamic view of the user&#x0027;s HMD. Here, we present a markerless calibration method integrating these two seamlessly into the mixed reality telepresence systems without disrupting the current workflow.
Revisiting Vicinal Risk Minimization for Partially Supervised Multi-Label Classification Under Data Scarcity

Due to the high human cost of annotation, it is nontrivial to curate a large-scale medical dataset that is fully labeled for all classes of interest. Instead, it would be convenient to collect multiple small partially labeled datasets from different matching sources, where the medical images may have only been annotated for a subset of classes of interest. This paper offers an empirical understanding of an under-explored problem, namely partially supervised multi-label classification (PSMLC), where a multi-label classifier is trained with only partially labeled medical images. In contrast to the fully supervised counterpart, the partial supervision caused by medical data scarcity has non-trivial negative impacts on the model performance. A potential remedy could be augmenting the partial labels. Though vicinal risk minimization (VRM) has been a promising solution to improve the generalization ability of the model, its application to PSMLC remains an open question. To bridge the methodological gap, we provide the first VRM-based solution to PSMLC. The empirical results also provide insights into future research directions on partially supervised learning under data scarcity.
EXTREM-EDGE—EXtensions To RISC-V for Energy-efficient ML inference at the EDGE of IoT

Artificial intelligence (AI) and machine learning (ML) have emerged as the fastest growing workloads ranging from applications like object detection, natural language processing and facial recognition to self-driving cars. The proliferation of these compute-intensive workloads resulted in numerous hardware accelerators to fill the gap between the performance and energy-efficiency requirements of AI applications and the capabilities of current architectures like CPU and GPU. In most cases these accelerators are specialized for a particular task, are costly to produce, require special programming tools, and can become obsolete as new ML algorithms are introduced. To solve these problems, we present EXTREM-EDGE, a hardware/software co-design approach to add custom extensions to the open-source RISC-V Instruction Set architecture (ISA) for designing a scalable and flexible ML processor architecture. EXTREM-EDGE augments the RISC-V processor with hardware AI functional units (AFU) along with ISA extensions which directly target these AFUs. EXTREM-EDGE is a system-level solution which is easy to program, enables royalty-free production and provides flexibility for future workloads. It enables the designers to quickly adapt to any hardware or ISA/software changes and allows the design-space exploration of various available hardware, instructions and software options. This enables a processor architecture which addresses the requirements of current AI/ML workloads, gives the flexibility to hot-swap AFUs when better hardware is available and scales with new AI instructions in response to rapidly evolving AI algorithms while providing a streamlined development flow for both hardware and software. EXTREM-EDGE provides 1.75x (MAC) to 17.63x (PIM VMM) performance improvements for a GEMV kernel and 1.41x (MAC) to 4.41x (PIM VMM) reductions in processor clock cycles for ResNet-8 neural network model from MLPerf Tiny benchmark depending upon the size of added accelerators and complexity of added instructions.
Using High Performance Approaches to Covid-19 Vaccines Sentiment Analysis

Coronavirus has emerged as challenge for the whole mankind causing illness worldwide. To eradicate the disease, global efforts are put increasing to develop its vaccine. In order to achieve the immunity against the virus, wide provision of vaccine is necessary. To make sure the distribution of vaccines, the sentiments of people for vaccines must be analyzed. Now-a-days, people share their thoughts, feelings and feedback about anything they experience on social media platforms. In this study, high performance approaches have been used for the analysis of the sentiments of people about vaccines. In this study, we have used the freely available data and applied pre-processing over it. We found out the polarity values of the tweets using TextBlob() function of Python and drew the wordclouds for positive, negative and neutral tweets. We used BERT model for understanding the people&#x2019;s feelings and feedback about vaccines. The model evaluation was performed using precision, recall and F measure. The BERT model achieved achieved 55 % &#x0026; 54 % precision, 69 % &#x0026; 85 % recall and 58 % &#x0026; 64 % F score for positive class and negative class respectively. Therefore, the use of artificial intelligence in social media analysis produce fruitful results while determining the people&#x2019;s attitude towards ant new trend, topic and any emergency situation. These methods helps to grow the vaccines campaigns timely by solving the people&#x2019;s concerns about vaccines.
Detection of plane in remote sensing images using super-resolution

The object detection of remote sensing image often has low accuracy and high missed or false detection rate due to the large number of small objects, instance level noise and cloud occlusion. In this paper, a new object detection model based on SRGAN and YOLOV3 is proposed, which is called SR-YOLO. It solves the problems of SRGAN network sensitivity to hyper-parameters and modal collapse. Meanwhile, The FPN network in YOLOv3 is replaced by PANet, shortened the distance between the lowest and the highest layers, and the SR-YOLO model has strong robustness and high detection ability by using the enhanced path to enrich the characteristics of each layer. The experimental results on the UCAS-High Resolution Aerial Object Detection Dataset showed SR-YOLO has achieved excellent performance. Compared with YOLOv3, the average precision (AP) of SR-YOLO increased from 92.35% to 96.13%, the log-average miss rate (MR-2) decreased from 22% to 14%, and the Recall rate increased from 91.36% to 95.12%.
Interpersonal Distance to a Speaking Avatar: Loudness Matters Irrespective of Contents

It is important for us to maintain appropriate interpersonal distance depending on situations in effective and safe communications. We aimed to investigate the effects of speech loudness and clarity on the interpersonal distance towards an avatar in a virtual environment. We found that the louder speech of the avatar made the distance between the participants and the avatar larger than the quiet speech, but the clarity of the speech did not significantly affect the distance. These results suggest that the perception of loudness modulates the interpersonal distance towards the virtual avatar to maintain the intimate equilibrium.
Multi-face detection and alignment using multiple kernels

Multi-face detection and alignment techniques under unlimited environment are challenging issues. In recent years, the demand for face detection and alignment techniques has increased in many areas, including automatic drive and security. However, some mainstream algorithms, such as the Multi-task Cascaded Convolutional Networks (MTCNN) algorithm cannot transfer to multi-face detection and alignment, which introduces new multi-size, multi-resolution, and multi-angle challenges. This paper proposes an improved algorithm—Multi-face-MTCNN for precise original face detection and alignment algorithm when there is an overlapping face scenario. We design two new network structures: Pixelfusion-MTCNN and Twoconv-MTCNN. Moreover, we propose new data augmentation method and optimized detection process which are applied in the Multi-face-MTCNN algorithm. The limitations associated with single-scale kernel size in MTCNN are solved to obtain a satisfactory performance. Compared to the MTCNN algorithm, experimental analysis of the FDDB dataset show that 1.766% improves Multi-face-MTCNN. Meanwhile, on the WIDER FACE verification benchmark, respectively, with regards to the three sub-datasets, the proposed algorithm’s performance is improved by 3.426%, 2.776%, and 21.576%. Besides, average alignment error of the left eye, right eye, nose, left mouth, and right mouth is performed on the proposed algorithm.
Leveraging Part-of-Speech Tagging Features and a Novel Regularization Strategy for Chinese Medical Named Entity Recognition

Chinese Medical Named Entity Recognition (Chinese-MNER) aims to identify potential entities and their categories from the unstructured Chinese medical text. Existing methods for this task mainly incorporate the dictionary knowledge on the basis of traditional BiLSTM-CRF or BERT architecture. However, the construction of high-quality dictionaries is typically time consuming and labor-intensive, which may also damage the robustness of NER models. What is more, the limited amount of annotated Chinese-MNER data can easily lead to the over-fitting problem while training. With the aim of dealing with the above problems, we put forward a BERT-BiLSTM-CRF model by integrating the part-of-speech (POS) tagging features and a Regularization method (BBCPR) for Chinese-MNER. In BBCPR, we first leverage a POS fusion layer to incorporate external syntax knowledge. Next, we design a novel REgularization mothod with Adversarial training and Dropout (READ) to improve the model robustness. Specifically, READ focuses on reducing the difference between the predictions of two sub-models through minimizing the bidirectional KL divergence between the adversarial output and original output distributions for the same sample. Comprehensive evaluations on two public data sets, namely, cMedQANER and cEHRNER from the Chinese Biomedical Language Understanding Evaluation benchmark (ChineseBLUE), demonstrate the superiority of our proposal in Chinese-MNER. In addition, ablation study shows that READ can effectively improve the model performance. Our proposal does well in exploring the technical terms and identifying the word boundary.
An Evaluation of Virtual Reality Maintenance Training for Industrial Hydraulic Machines

Virtual reality applications for industrial training have widespread benefits for simulating various scenarios and conditions. We present an empirical evaluation of VR training approach using kinesthetics learning strategy in industrial maintenance training, specifically the hydraulic manufacturing industry. Through our collaboration with a leading industry partner, a remote multi-user training platform using head-mounted display was designed and implemented. We present the evaluation of the platform with two diverse cohorts of novice users and industry contractors, in comparison to traditional training using slides, photos, and videos. The results show that VR training is engaging and effective in boosting trainee&#x2019;s confidence, especially for novice users. Our studies highlight the impact of virtual reality training on trainee experience, performance, and skills transfer, with reflections on the differences between novice and industry trainees.
Towards Attack to MemGuard with Nonlocal-Means Method

An adversarial example is the weakness of the machine learning (ML), and it can be utilized as the tool to defend against the inference attacks launched by ML classifiers. Jia et al. proposed MemGuard, which applied the idea of adversarial example to defend against membership inference attack. In a membership inference attack, the attacker attempts to infer whether a particular sample is in the training set of the target classifier, which may be a software or a service whose model parameters are unknown to the attacker. MemGuard does not tamper the training process of the target classifier, meanwhile achieving better tradeoff between the privacy and utility loss. However, many defenses of the adversarial example have been proposed, which decreases the effectiveness of the adversarial example. Inspired by the defenses of the adversarial example, we try to attack the MemGuard. As a result, we utilize the nonlocal-means method to do the attack by using the inherent relationship between neighbor entries to remove the added noise. Due to the low dimensionality of the confidence score vector, our attack avoids the high computation overhead of the nonlocal-means method. Besides, we use practical datasets to test our attack, and the experimental results demonstrate the effectiveness of our attack.
Knowing the Partner's Objective Increases Embodiment towards a Limb Controlled by the Partner

We have developed a joint avatar of which two participants simultaneously control left and right-side limbs in first-person view in a virtual environment. We aimed to investigate whether having a common objective with a partner affects sense of embodiment towards a limb controlled by the partner. Participants performed reaching tasks using the joint avatar. We found that the embodiment towards the arm controlled by the partner was significantly higher when the participant dyads shared a common objective or when they were allowed to see their partner&#x0027;s goal, compared to when their partner&#x0027;s goal was unknown to them.
A Robust Color Edge Detection Algorithm Based on the Quaternion Hardy Filter

This paper presents a robust filter called the quaternion Hardy filter (QHF) for color image edge detection. The QHF can be capable of color edge feature enhancement and noise resistance. QHF can be used flexibly by selecting suitable parameters to handle different levels of noise. In particular, the quaternion analytic signal, which is an effective tool in color image processing, can also be produced by quaternion Hardy filtering with specific parameters. Based on the QHF and the improved Di Zenzo gradient operator, a novel color edge detection algorithm is proposed; importantly, it can be efficiently implemented by using the fast discrete quaternion Fourier transform technique. From the experimental results, we conclude that the minimum PSNR improvement rate is 2.3% and the minimum SSIM improvement rate is 30.2% on the CSEE database. The experiments demonstrate that the proposed algorithm outperforms several widely used algorithms.
S-NER: A Concise and Efficient Span-Based Model for Named Entity Recognition

Named entity recognition (NER) is a task that seeks to recognize entities in raw texts and is a precondition for a series of downstream NLP tasks. Traditionally, prior NER models use the sequence labeling mechanism which requires label dependency captured by the conditional random fields (CRFs). However, these models are prone to cascade label misclassifications since a misclassified label results in incorrect label dependency, and so some following labels may also be misclassified. To address the above issue, we propose S-NER, a span-based NER model. To be specific, S-NER first splits raw texts into text spans and regards them as candidate entities; it then directly obtains the types of spans by conducting entity type classifications on span semantic representations, which eliminates the requirement for label dependency. Moreover, S-NER has a concise neural architecture in which it directly uses BERT as its encoder and a feed-forward network as its decoder. We evaluate S-NER on several benchmark datasets across three domains. Experimental results demonstrate that S-NER consistently outperforms the strongest baselines in terms of F1-score. Extensive analyses further confirm the efficacy of S-NER.
All Shook Up: The Impact of Floor Vibration in Symmetric and Asymmetric Immersive Multi-user VR Gaming Experiences

This paper investigates the influence of floor-vibration tactile feedback on immersed users. Under symmetric and asymmetric tactile sensory cue conditions, we explore how multi-user Virtual Reality (VR) experiences are impacted by these cues in terms of illusion and coherence. Based on the reported positive impact of tactile cues in solo VR experiences, we posit that if context-matched perceptual tactile feedback is exchanged between users, they will report a significantly enhanced VR experience compared to not receiving the sensory stimuli, even within the same immersive VR experience. With our custom-built, computer-controlled vibration floor, we implemented a cannonball shooting game for two physically-separated players. In the VR game, the two players shoot cannonballs to destroy their opponent&#x2019;s protective wall and cannon, while the programmed floor platform generates vertical vibrations depending on the experimental condition. We used a mixed-factorial design with four conditions for each pair of participants: 1) both A and B had vibration, and 2) neither A nor B had vibration (the Symmetric group), or 3) A had vibration, but B did not, and 4) B had vibration, but A did not (the Asymmetric group). We collected subjective and objective data for variables previously shown to be related to levels of illusion, coherence, and usability, including Presence, Co-Presence, Social Presence, Plausibility Illusion, Engagement, Embodiment, Coherence, Gaming Performance, and Overall Preference. A total of 39 pairs of participants were involved in the study. We found statistically significant differences for the vibration conditions on Co-Presence, Social Presence, Engagement, and Coherence, and for the symmetric conditions on the Plausibility Illusion and Coherence, but only with trivial or small effect sizes. The results indicate that vibration provided to a pair of game players in immersive VR can significantly enhance the VR experience, but sensory symmetry does not guarantee improved gaming performance.
Super-resolution reconstruction of biometric features recognition based on manifold learning and deep residual network

In daily life, face information has the characteristics of uniqueness and universality. However, in a real-world scene, the image information of the face acquired by the acquisition device often contains noises such as blurring and sharpening. As such, super-resolution reconstruction of face features recognition based on manifold learning is proposed in this paper.We reconstruct low-resolution facial expression images, introduce a simplified residual block network and manifold learning, and propose joint supervision through a new hybrid loss function, which not only retains the color and characteristics of the image, but also retains the high-frequency information. The ResNet50 network uses the weight feature of information entropy to optimize the information of the pooling layer, and the esNet50 network uses the improved PSO algorithm to optimize the initial weight of the error back-propagation phase.In the case of inputting extremely low resolution (6 × 6) facial expression images, the accuracy rate is increased by 9.091%. The accuracy of the high-resolution facial expressions after reconstruction with a size of 12×12 is 96.970%. The accuracy rate for happy expressions is 100%, the accuracy rate for anger, disgust, sadness, and surprise recognition is 97%, the accuracy rate for contempt is 94%, and the accuracy rate for fear is 88%.The experimental results verify the feasibility and superiority of the system, and effectively improve the accuracy of low-resolution facial expressions.