Reviewerly/exHarmony huggingface dataset:
Evaluating the effect of multi-sensory stimulations on simulator sickness and sense of presence during HMD-mediated VR experience

Some lines of evidence have shown that sensory input, especially related to vestibular and somatosensory stimulation, may reduce the symptoms related to simulator sickness and increase the sense of presence in VR. The present study aims at understanding how mechanical vibration and auditory stimulation can be used to improve user experience in the context of VR mediated by head-mounted displays. Four different groups comprising a total of 80 participants were tested under different conditions of sensory input (visual and vibration, visual-auditory, combined visual-auditory and vibratory, and visual only), during a VR roller-coaster experience. No significant differences in simulator sickness were found between the groups exposed to seat vibration and/or audio. However, sense of presence showed to be increased when vibratory stimuli were included. Post-hoc analyses showed that female users but not male ones, experienced an increase of sense of presence when vibratory stimulation was used. Practitioner summary: The study showed that including sound or vibration stimulation during VR experience does not reduce simulator sickness. However, sense of presence is promoted by vibratory stimulation. Post-hoc analyses showed that female users experienced an increase of sense of presence by vibratory stimulation, but not male ones.
HandPainter - 3D Sketching in VR with Hand-based Physical Proxy

3D sketching in virtual reality (VR) enables users to create 3D virtual objects intuitively and immersively. However, previous studies showed that mid-air drawing may lead to inaccurate sketches. To address this issue, we propose to use one hand as a canvas proxy and the index finger of the other hand as a 3D pen. To this end, we first perform a formative study to compare two-handed interaction with tablet-pen interaction for VR sketching. Based on the findings of this study, we design HandPainter, a VR sketching system which focuses on the direct use of two hands for 3D sketching without requesting any tablet, pen, or VR controller. Our implementation is based on a pair of VR gloves, which provide hand tracking and gesture capture. We devise a set of intuitive gestures to control various functionalities required during 3D sketching, such as canvas panning and drawing positioning. We show the effectiveness of HandPainter by presenting a number of sketching results and discussing the outcomes of a user study-based comparison with mid-air drawing and tablet-based sketching tools.
Documented: Embedding Information onto and Retrieving Information from 3D Printed Objects

Documentation for DIY tasks serve as codified project knowledge and help makers reach new understandings and appreciations for the artifact. Engaging in reflective processes using the documentation can be challenging when it comes to physical objects as the documentation and the artifact exist separately. We hypothesize that spatially associating the documentation information with the artifact can provide richer contextualization to reflect upon the artifact and design process. We implemented and evaluated Documented, a web application that helps makers associate documentation to 3D printed objects. Information can be embedded using printed tags spatially placed on the model and accessed using mobile AR. Our study highlights the different strategies participants had for organizing, embedding, and retrieving information. Informed by our results, we discuss how the coupling of the documentation and the artifact can support reflection and identify potential barriers that need further investigation.
Ontology Creation Model based on Attention Mechanism for a Specific Business Domain

Ontology can describe real-world objects and relationships. It can be used for searching, web mining, and semantic analysis. However, creating the ontology for a different domain is a labor-intensive and computationally expensive task. This work introduces a new model to automate ontology creation for the real estate domain based on a natural language processing approach. The new model can save significant time and human labor in ontology creation in several domains. Also, the new model can enhance retrieval information systems in advanced semantic web search engines.
CodeShovel: Constructing Method-Level Source Code Histories

Source code histories are commonly used by developers and researchers to reason about how software evolves. Through a survey with 42 professional software developers, we learned that developers face significant mismatches between the output provided by developers' existing tools for examining source code histories and what they need to successfully complete their historical analysis tasks. To address these shortcomings, we propose CodeShovel, a tool for uncovering method histories that quickly produces complete and accurate change histories for 90% methods (including 97% of all method changes) outperforming leading tools from both research (e.g, FinerGit) and practice (e.g., IntelliJ / git log). CodeShovel helps developers to navigate the entire history of source code methods so they can better understand how the method evolved. A field study on industrial code bases with 16 industrial developers confirmed our empirical findings of CodeShovel's correctness, low runtime overheads, and additionally showed that the approach can be useful for a wide range of industrial development tasks.
Multi-objective Cuckoo Search-based Streaming Feature Selection for Multi-label Dataset

The feature selection method is the process of selecting only relevant features by removing irrelevant or redundant features amongst the large number of features that are used to represent data. Nowadays, many application domains especially social media networks, generate new features continuously at different time stamps. In such a scenario, when the features are arriving in an online fashion, to cope up with the continuous arrival of features, the selection task must also have to be a continuous process. Therefore, the streaming feature selection based approach has to be incorporated, i.e., every time a new feature or a group of features arrives, the feature selection process has to be invoked. Again, in recent years, there are many application domains that generate data where samples may belong to more than one classes called multi-label dataset. The multiple labels that the instances are being associated with, may have some dependencies amongst themselves. Finding the co-relation amongst the class labels helps to select the discriminative features across multiple labels. In this article, we develop streaming feature selection methods for multi-label data where the multiple labels are reduced to a lower-dimensional space. The similar labels are grouped together before performing the selection method to improve the selection quality and to make the model time efficient. The multi-objective version of the cuckoo search-based approach is used to select the optimal feature set. The proposed method develops two versions of the streaming feature selection method: <?TeX $(1$?> ) when the features arrive individually and <?TeX $(2$?> ) when the features arrive in the form of a batch. Various multi-label datasets from various domains such as text, biology, and audio have been used to test the developed streaming feature selection methods. The proposed methods are compared with many previous feature selection methods and from the comparison, the superiority of using multiple objectives and label co-relation in the feature selection process can be established.
StoryMap: Using Social Modeling and Self-Modeling to Support Physical Activity Among Families of Low-SES Backgrounds

Physical activity (PA) is crucial for reducing the risk of obesity, an epidemic that disproportionately burdens families of low-socioeconomic status (SES). While fitness tracking tools can increase PA awareness, more work is needed to examine (1) how such tools can help people benefit from their social environment, and (2) how reflections can help enhance PA attitudes. We investigated how fitness tracking tools for families can support social modeling and self-modeling (through reflection), two critical processes in Social Cognitive Theory. We developed StoryMap, a novel fitness tracking app for families aimed at supporting both modes of modeling. Then, we conducted a five-week qualitative study evaluating StoryMap with 16 low-SES families. Our findings contribute an understanding of how social and self-modeling can be implemented in fitness tracking tools and how both modes of modeling can enhance key PA attitudes: self-efficacy and outcome expectations. Finally, we propose design recommendations for social personal informatics tools.
Blockchain Empowered Reliable Federated Learning by Worker Selection: A Trustworthy Reputation Evaluation Method

Federated learning is a distributed machine learning framework that enables distributed model training with local datasets, which can effectively protect the data privacy of workers (i.e., intelligent edge nodes). The majority of federated learning algorithms assume that the workers are trusted and voluntarily participate in the cooperative model training process. However, the situation in practical application is not consistent with this. There are many challenges such as worker selection schemes for participating workers, which hamper the widespread adoption of federated learning. The existing research about worker selection scheme focused on multi-weight subjective logic model to calculate reputation value and adopted contract theory to motivate workers, which may exist subjective judgmental factors and unfair profit distribution. To address above challenges, we calculate the reputation value by model quality parameters to evaluate the reliability of workers. Blockchain is designed to store historical reputation value that realized tamperresistance and non-repudiation. Numerical results indicate that the worker selection scheme can improve the accuracy of the model and accelerate the model convergence.
A novel image encryption using random matrix affine cipher and the chaotic maps

Security issues are at its peak by the expanding pace of innovation and the utilization of interactive media over the internet. To overcome such security issues, cryptography, steganography and watermarking methods are utilized eventually. In this work, we have fabricated a lossless cryptographic technique for image encryption using random matrix affine cipher (RMAC) associated with the chaotic maps of 2D Hénon (HN) and logistic. The method gives multi-layer security to forestall unauthorized access during transmission over the public networks. This method of encryption is fast with a high throughput of correlated data and works efficiently when compared to some available existing schemes. Numerical simulations and performance analysis insured the security, validation and stalwartness of the presented scheme. Further, the proposed scheme can be efficiently used in practice for secure transmission in wireless applications and small system devices.
FC<sup>2</sup>RN: A Fully Convolutional Corner Refinement Network for Accurate Multi-Oriented Scene Text Detection

Accurate detection of multi-oriented text that accounts for a large proportion in real practice is of great significance. The performance has improved rapidly on common benchmarks in recent years. However, dense long text case and the quality of detection are easy to be overlooked. Direct regression may produce low-quality and incomplete detections due to the constrain of the receptive field; proposal-based methods could alleviate this but might introduce redundant context due to RoI operation, degrading the performance. To address the dilemma, a novel proposed corner-aware convolution in which the sampling positions tightly cover the text area is utilized to encode an initial corner prediction into the feature maps, which can be further used to produce a refined corner prediction. We embed the proposed module into an anchor-free baseline model, leading to a simple and effective fully convolutional corner refinement network (FC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RN). Experimental results on four public datasets including MSRATD500, ICDAR2015, RCTW-17, and COCO-Text demonstrate that FC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> RN can outperform state-of-the-art methods.
Phonetroller: Visual Representations of Fingers for Precise Touch Input with Mobile Phones in VR

Smartphone touch screens are potentially attractive for interaction in virtual reality (VR). However, the user cannot see the phone or their hands in a fully immersive VR setting, impeding their ability for precise touch input. We propose mounting a mirror above the phone screen such that the front-facing camera captures the thumbs on or near the screen. This enables the creation of semi-transparent overlays of thumb shadows and inference of fingertip hover points with deep learning, which help the user aim for targets on the phone. A study compares the effect of visual feedback on touch precision in a controlled task and qualitatively evaluates three example applications demonstrating the potential of the technique. The results show that the enabled style of feedback is effective for thumb-size targets, and that the VR experience can be enriched by using smartphones as VR controllers supporting precise touch input.
Mining Architecture Tactics and Quality Attributes knowledge in Stack Overflow

Context: Architecture Tactics (ATs) are architectural building blocks that provide general architectural solutions for addressing Quality Attributes (QAs) issues. Mining and analyzing QA-AT knowledge can help the software architecture community better understand architecture design. However, manually capturing and mining this knowledge is labor-intensive and difficult. Objective: Using Stack Overflow (SO) as our source, our main goals are to effectively mine such knowledge; and to have some sense of how developers use ATs with respect to QA concerns from related discussions. Methods: We applied a semi-automatic dictionary-based mining approach to extract the QA-AT posts in SO. With the mined QA-AT posts, we identified the relationships between ATs and QAs. Results: Our approach allow us to mine QA-AT knowledge effectively with an F-measure of 0.865 and Performance of 82.2%. Using this mining approach, we are able to discover architectural synonyms of QAs and ATs used by designers, from which we discover how developers apply ATs to address quality requirements. Conclusions: We make two contributions in this work: First, we demonstrated a semi-automatic approach to mine ATs and QAs from SO posts; Second, we identified little-known design relationships between QAs and ATs and grouped architectural design considerations to aid architects make architecture tactics design decisions.
Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs

While Generative Adversarial Networks (GANs) show increasing performance and the level of realism is becoming indistinguishable from natural images, this also comes with high demands on data and computation. We show that state-of-the-art GAN models – such as they are being publicly released by researchers and industry – can be used for a range of applications beyond unconditional image generation. We achieve this by an iterative scheme that also allows gaining control over the image generation process despite the highly non-linear latent spaces of the latest GAN models. We demonstrate that this opens up the possibility to re-use state-of-the-art, difficult to train, pre-trained GANs with a high level of control even if only black-box access is granted. Our work also raises concerns and awareness that the use cases of a published GAN model may well reach beyond the creators’ intention, which needs to be taken into account before a full public release. Code is available at https://github.com/a514514772/hijackgan.
Deepfake Video Detection Based on Spatial, Spectral, and Temporal Inconsistencies Using Multimodal Deep Learning

Authentication of digital media has become an ever-pressing necessity for modern society. Since the introduction of Generative Adversarial Networks (GANs), synthetic media has become increasingly difficult to identify. Synthetic videos that contain altered faces and/or voices of a person are known as deepfakes and threaten trust and privacy in digital media. Deep-fakes can be weaponized for political advantage, slander, and to undermine the reputation of public figures. Despite imperfections of deepfakes, people struggle to distinguish between authentic and manipulated images and videos. Consequently, it is important to have automated systems that accurately and efficiently classify the validity of digital content. Many recent deepfake detection methods use single frames of video and focus on the spatial information in the image to infer the authenticity of the video. Some promising approaches exploit the temporal inconsistencies of manipulated videos; however, research primarily focuses on spatial features. We propose a hybrid deep learning approach that uses spatial, spectral, and temporal content that is coupled in a consistent way to differentiate real and fake videos. We show that the Discrete Cosine transform can improve deepfake detection by capturing spectral features of individual frames. In this work, we build a multimodal network that explores new features to detect deepfake videos, achieving 61.95% accuracy on the Facebook Deepfake Detection Challenge (DFDC) dataset.
Detection and Isolation of Sensor Attacks for Autonomous Vehicles: Framework, Algorithms, and Validation

This paper investigates the cyber-security problem for autonomous vehicles under sensor attacks. In particular, a model-based framework is proposed which can detect sensor attacks and identify their sources in order to achieve the secure localization of self-driving vehicles. To ensure robustness of the vehicle against cyber-attacks, sensor redundancy is introduced, that is to deploy multiple sensors, each of which provides real-time pose observations of the vehicle. A bank of attack detectors is developed to capture anomalies in each sensor measurement, which is a combination of an extended Kalman filter (EKF) and a cumulative sum (CUSUM) discriminator. EKFs are employed to estimate the vehicle position and orientation recursively, while each CUSUM discriminator is designed to analyze the residual generated by its combined EKF to detect the possible deviation of the sensor measurement from the expected pose derived according to the mathematical model of the vehicle. To monitor the inconsistency amongst multiple sensor measurements, an auxiliary detector is introduced which fuses observations from multiple sensors. Based on the results of all the detectors, a rule-based isolation scheme is developed to identify the source anomalous sensor. The effectiveness of our proposed framework has been demonstrated on real vehicle data.
vMirror: Enhancing the Interaction with Occluded or Distant Objects in VR with Virtual Mirrors

Interacting with out of reach or occluded VR objects can be cumbersome. Although users can change their position and orientation, such as via teleporting, to help observe and select, doing so frequently may cause loss of spatial orientation or motion sickness. We present vMirror, an interactive widget leveraging reflection of mirrors to observe and select distant or occluded objects. We first designed interaction techniques for placing mirrors and interacting with objects through mirrors. We then conducted a formative study to explore a semi-automated mirror placement method with manual adjustments. Next, we conducted a target-selection experiment to measure the effect of the mirror's orientation on users' performance. Results showed that vMirror can be as efficient as direct target selection for most mirror orientations. We further compared vMirror with teleport technique in a virtual treasure hunt game and measured participants' task performance and subjective experiences. Finally, we discuss vMirorr user experience and present future directions.
Evaluating SZZ Implementations Through a Developer-Informed Oracle

The SZZ algorithm for identifying bug-inducing changes has been widely used to evaluate defect prediction techniques and to empirically investigate when, how, and by whom bugs are introduced. Over the years, researchers have proposed several heuristics to improve the SZZ accuracy, providing various implementations of SZZ. However, fairly evaluating those implementations on a reliable oracle is an open problem: SZZ evaluations usually rely on (i) the manual analysis of the SZZ output to classify the identified bug-inducing commits as true or false positives; or (ii) a golden set linking bug-fixing and bug-inducing commits. In both cases, these manual evaluations are performed by researchers with limited knowledge of the studied subject systems. Ideally, there should be a golden set created by the original developers of the studied systems. We propose a methodology to build a "developer-informed" oracle for the evaluation of SZZ variants. We use Natural Language Processing (NLP) to identify bug-fixing commits in which developers explicitly reference the commit(s) that introduced a fixed bug. This was followed by a manual filtering step aimed at ensuring the quality and accuracy of the oracle. Once built, we used the oracle to evaluate several variants of the SZZ algorithm in terms of their accuracy. Our evaluation helped us to distill a set of lessons learned to further improve the SZZ algorithm.
Internet-based language production research with overt articulation: Proof of concept, challenges, and practical advice

Language production experiments with overt articulation have thus far only scarcely been conducted online, mostly due to technical difficulties related to measuring voice onset latencies. Especially the poor audiovisual synchrony in web experiments (Bridges et al. 2020) is a challenge to time-locking stimuli and participants' spoken responses. We tested the viability of conducting language production experiments with overt articulation in online settings using the picture-word interference paradigm - a classic task in language production research. In three pre-registered experiments (N = 48 each), participants named object pictures while ignoring visually superimposed distractor words. We implemented a custom voice recording option in two different web experiment builders and recorded naming responses in audio files. From these stimulus-locked audio files, we extracted voice onset latencies offline. In a control task, participants classified the last letter of a picture name as a vowel or consonant via button-press, a task that shows comparable semantic interference effects. We expected slower responses when picture and distractor word were semantically related compared to unrelated, independently of task. This semantic interference effect is robust, but relatively small. It should therefore crucially depend on precise timing. We replicated this effect in an online setting, both for button-press and overt naming responses, providing a proof of concept that naming latency - a key dependent variable in language production research - can be reliably measured in online experiments. We discuss challenges for online language production research and suggestions of how to overcome them. The scripts for the online implementation are made available.