Reviewerly/exHarmony huggingface dataset:
Reducing Virtual Reality Sickness for Cyclists in VR Bicycle Simulators

Virtual Reality (VR) bicycle simulations aim to recreate the feeling of riding a bicycle and are commonly used in many application areas. However, current solutions still create mismatches between the visuals and physical movement, which causes VR sickness and diminishes the cycling experience. To reduce VR sickness in bicycle simulators, we conducted two controlled lab experiments addressing two main causes of VR sickness: (1) steering methods and (2) cycling trajectory. In the first experiment (N = 18) we compared handlebar, HMD, and upper-body steering methods. In the second experiment (N = 24) we explored three types of movement in VR (1D, 2D, and 3D trajectories) and three countermeasures (airflow, vibration, and dynamic Field-of-View) to reduce VR sickness. We found that handlebar steering leads to the lowest VR sickness without decreasing cycling performance and airflow suggests to be the most promising method to reduce VR sickness for all three types of trajectories.
Learn from structural scope: Improving aspect-level sentiment analysis with hybrid graph convolutional networks

Aspect-level sentiment analysis aims to determine the sentiment polarity towards a specific target in a sentence. The main challenge of this task is to effectively model the relation between targets and sentiments so as to filter out noisy opinion words from irrelevant targets. Most recent efforts capture relations through target-sentiment pairs or opinion spans from a word-level or phrase-level perspective. Based on the observation that targets and sentiments essentially establish relations following the grammatical hierarchy of phrase-clause-sentence structure, it is hopeful to exploit comprehensive syntactic information for better guiding the learning process. Therefore, we introduce the concept of Scope, which outlines a structural text region related to a specific target. To jointly learn structural Scope and predict the sentiment polarity, we propose a hybrid graph convolutional network (HGCN) to synthesize information from constituency tree and dependency tree, exploring the potential of linking two syntax parsing methods to enrich the representation. Experimental results on four public datasets illustrate that our HGCN model outperforms current state-of-the-art baselines.
More-than-human Concepts, Methodologies, and Practices in HCI

The last decade has witnessed the expansion of design space to include the epistemologies and methodologies of more-than-human design (MTHD). Design researchers and practitioners have been increasingly studying, designing for, and designing with nonhumans. This panel will bring together HCI experts who work on MTHD with different nonhumans as their subjects. Panelists will engage the audience through discussion of their shared and diverging visions, perspectives, and experiences, and through suggestions for opportunities and challenges for the future of MTHD. The panel will provoke the audience into reflecting on how the emergence of MTHD signals a paradigm shift in HCI and human-centered design, what benefits this shift might bring and whether MTH should become the mainstream approach, as well as how to involve nonhumans in design and research.
Lymph Node Metastasis Prediction From Whole Slide Images With Transformer-Guided Multiinstance Learning and Knowledge Transfer

The gold standard for diagnosing lymph node metastasis of papillary thyroid carcinoma is to analyze the whole slide histopathological images (WSIs). Due to the large size of WSIs, recent computer-aided diagnosis approaches adopt the multi-instance learning (MIL) strategy and the key part is how to effectively aggregate the information of different instances (patches). In this paper, a novel transformer-guided framework is proposed to predict lymph node metastasis from WSIs, where we incorporate the transformer mechanism to improve the accuracy from three different aspects. First, we propose an effective transformer-based module for discriminative patch feature extraction, including a lightweight feature extractor with a pruned transformer (Tiny-ViT) and a clustering-based instance selection scheme. Next, we propose a new Transformer-MIL module to capture the relationship of different discriminative patches with sparse distribution on WSIs and better nonlinearly aggregate patch-level features into the slide-level prediction. Considering that the slide-level annotation is relatively limited to training a robust Transformer-MIL, we utilize the pathological relationship between the primary tumor and its lymph node metastasis and develop an effective attention-based mutual knowledge distillation (AMKD) paradigm. Experimental results on our collected WSI dataset demonstrate the efficiency of the proposed Transformer-MIL and attention-based knowledge distillation. Our method outperforms the state-of-the-art methods by over 2.72% in AUC (area under the curve).
Speaking vs. listening? Balance conversation attributes of voice assistants for better voice marketing

Artificial Intelligence (AI) is shaping marketing in an unprecedented way. Empowered by AI, voice assistants are increasingly capable of speaking and listening like humans, offering a great opportunity for a new marketing approach - voice marketing. This research examines how conversation attributes of voice assistants determine consumer trust and intention to engage in voice shopping. Using a sequential mixed-method design, three studies consistently show that consumers perceive the speaking attribute of voice assistants as more human-like than the listening attribute. We find that such incongruency between the two conversation attributes can undermine consumers' trust in voice assistants, leading to reduced willingness to accept product recommendations from voice assistants and shop via voice assistants, which would hamper the development of voice marketing. Accordingly, this research suggests that AI giants with strong technological strength and capital support should distribute more resources to advance the underlying technologies enabling human-like listening (e.g., natural language understanding and voice recognition). But for AI startups with limited financing ability and technical talents, they may consider appropriately reducing investments in the underlying technologies enabling human-like speaking (e.g., natural language generation and voice synthesis) to enhance the congruency level between the conversation attributes of voice assistants.
SUM: Serialized Updating and Matching for text-based person retrieval

The central problem of text-based person retrieval is how to properly bridge the gap between heterogeneous cross-modal data. Many of the previous works contrive to learn a latent common space to bridge the modality gap and extract modality-invariant feature vectors. Within these methods, the common space mapping and cross-modal information matching operations are conducted in a one-off manner, which aims to extract sufficient discriminative clues from the high-dimensional multi-modal data at first glance, but it is inconsistent with the fact that humans usually follow a step-by-step process to properly recognize and match two objects. Intuitively, the large heterogeneity gap between multi-modal data can be better bridged by gradually analyzing the complex cross-modal relationships. In this paper, we propose a Serialized Updating and Matching (SUM) method for text-based person retrieval to bridge the heterogeneity gap between cross-modal data in a step-by-step manner. The core component of SUM is the proposed Memory Gating Modules (MGM), which can be stacked to gradually update and match features extracted from visual/textual modalities. To fully excavate the correlations lie within multi-granular cross-modal data, two variants are designed to care for both global and fine-grain local information, namely, Global Memory Gating Module (GMGM) and Fine-grained Memory Gating Module (FMGM) with which the updating rate of information at each step is dynamically determined after observing the feature in opposite modality. Moreover, SUM can be flexibly utilized as an add-on to any multi-granular text-based person retrieval methods to further improve the performance. We evaluate our proposed method on two text-based person retrieval datasets CUHK-PEDES and RSTPReid along with two general cross-modal retrieval datasets Flickr8K and Flickr30K to see its generalization ability. Experimental results present that the proposed SUM outperforms existing methods and achieves the state-of-the-art performance.
You Complete Me: Human-AI Teams and Complementary Expertise

People consider recommendations from AI systems in diverse domains ranging from recognizing tumors in medical images to deciding which shoes look cute with an outfit. Implicit in the decision process is the perceived expertise of the AI system. In this paper, we investigate how people trust and rely on an AI assistant that performs with different levels of expertise relative to the person, ranging from completely overlapping expertise to perfectly complementary expertise. Through a series of controlled online lab studies where participants identified objects with the help of an AI assistant, we demonstrate that participants were able to perceive when the assistant was an expert or non-expert within the same task and calibrate their reliance on the AI to improve team performance. We also demonstrate that communicating expertise through the linguistic properties of the explanation text was effective, where embracing language increased reliance and distancing language reduced reliance on AI.
Systematic literature review on smart contracts in the construction industry: Potentials, benefits, and challenges

Abstract The development of digital transformation in the construction industry has led to the increasing adoption of smart contracts. As programmable applications to automatically write, verify, and enforce transaction conditions, smart contracts can be used in different areas mainly to improve automation level, information security, and built digital environment enhancement. However, the smart contract is commonly mentioned as a blockchain appendage, while its unique connotation and value in the construction industry have not been recognized. Therefore, this study carries out a systematic review based on 81 research articles published from 2014 to 2021 on smart contract applications in construction to explore and highlight their potentials under domain-specific requirements. Results are analyzed according to research type categorization and domain codification. Eight research domains are identified, where the three most highly explored domains are contract and payment, supply chain and logistics, and information management. The integration of smart contracts with other innovative concepts and advanced technologies is analyzed. The applicability, benefits, and challenges of smart contract applications regarding different research domains are discussed.
YOLOv5-Based Vehicle Detection Method for High-Resolution UAV Images

To solve the feature loss caused by the compression of high-resolution images during the normalization stage, an adaptive clipping algorithm based on the You Only Look Once (YOLO) object detection algorithm is proposed for the data preprocessing and detection stage. First, a high-resolution training dataset is augmented with the adaptive clipping algorithm. Then, a new training set is generated to retain the detailed features that the object detection network needs to learn. During the network detection process, the image is detected in chunks via the adaptive clipping algorithm, and the coordinates of the detection results are merged by position mapping. Finally, the chunked detection results are collocated with the global detection results and outputted. The improved YOLO algorithm is used to conduct experiments comparing this algorithm with the original algorithm for the detection of test set vehicles. The experimental results show that compared with the original YOLO object detection algorithm, the precision of our algorithm is increased from 79.5% to 91.9%, the recall is increased from 44.2% to 82.5%, and the mAP@0.5 is increased from 47.9% to 89.6%. The application of the adaptive clipping algorithm in the vehicle detection process effectively improves the performance of the traditional object detection algorithm.
Deep learning from multiple experts improves identification of amyloid neuropathologies

Pathologists can label pathologies differently, making it challenging to yield consistent assessments in the absence of one ground truth. To address this problem, we present a deep learning (DL) approach that draws on a cohort of experts, weighs each contribution, and is robust to noisy labels. We collected 100,495 annotations on 20,099 candidate amyloid beta neuropathologies (cerebral amyloid angiopathy (CAA), and cored and diffuse plaques) from three institutions, independently annotated by five experts. DL methods trained on a consensus-of-two strategy yielded 12.6-26% improvements by area under the precision recall curve (AUPRC) when compared to those that learned individualized annotations. This strategy surpassed individual-expert models, even when unfairly assessed on benchmarks favoring them. Moreover, ensembling over individual models was robust to hidden random annotators. In blind prospective tests of 52,555 subsequent expert-annotated images, the models labeled pathologies like their human counterparts (consensus model AUPRC = 0.74 cored; 0.69 CAA). This study demonstrates a means to combine multiple ground truths into a common-ground DL model that yields consistent diagnoses informed by multiple and potentially variable expert opinions.
Robust Disentangled Variational Speech Representation Learning for Zero-Shot Voice Conversion

Traditional studies on voice conversion (VC) have made progress with parallel training data and known speakers. Good voice conversion quality is obtained by exploring better alignment modules or expressive mapping functions. In this study, we investigate zero-shot VC from a novel perspective of self-supervised disentangled speech representation learning. Specifically, we achieve the disentanglement by balancing the information flow between global speaker representation and time-varying content representation in a sequential variational autoencoder (VAE). A zero-shot voice conversion is performed by feeding an arbitrary speaker embedding and content embeddings to the VAE decoder. Besides that, an on-the-fly data augmentation training strategy is applied to make the learned representation noise invariant. On TIMIT and VCTK datasets, we achieve state-of-the-art performance on both objective evaluation, i.e., speaker verification (SV) on speaker embedding and content embedding, and subjective evaluation, i.e., voice naturalness and similarity, and remains to be robust even with noisy source/target utterances.
Multi-Modality Self-Distillation for Weakly Supervised Temporal Action Localization

As a challenging task of high-level video understanding, Weakly-supervised Temporal Action Localization (WTAL) has attracted increasing attention in recent years. However, due to the weak supervisions of whole-video classification labels, it is challenging to accurately determine action instance boundaries. To address this issue, pseudo-label-based methods [Alwassel et al. (2019), Luo et al. (2020), and Zhai et al. (2020)] were proposed to generate snippet-level pseudo labels from classification results. In spite of the promising performance, these methods hardly take full advantages of multiple modalities, i.e., RGB and optical flow sequences, to generate high quality pseudo labels. Most of them ignored how to mitigate the label noise, which hinders the capability of the network on learning discriminative feature representations. To address these challenges, we propose a Multi-Modality Self-Distillation (MMSD) framework, which contains two single-modal streams and a fused-modal stream to perform multi-modality knowledge distillation and multi-modality self-voting. On the one hand, multi-modality knowledge distillation improves snippet-level classification performance by transferring knowledge between single-modal streams and a fused-modal stream. On the other hand, multi-modality self-voting mitigates the label noise in a modality voting manner according to the reliability and complementarity of the streams. Experimental results on THUMOS14 and ActivityNet1.3 datasets demonstrate the effectiveness of our method and superior performance over state-of-the-art approaches. Our code is available at https://github.com/LeonHLJ/MMSD.
Visually meaningful image encryption algorithm based on digital signature

Traditional image encryption algorithms transform a plain image into a noise-like image. To lower the chances for the encrypted image being detected by the attacker during the image transmission, a visually meaningful image encryption scheme is suggested to hide the encrypted image using another carrier image. This paper proposes a visually meaningful encrypted image algorithm that hides a secret image and a digital signature which provides authenticity and confidentiality. The recovered digital signature is used for the purpose of identity authentication while the secret image is encrypted to protect its confidentiality. Least Significant Bit (LSB) method to embed signature on the encrypted image and Lifting Wavelet Transform (LWT) to generate a visually meaningful encrypted image are designed. The proposed algorithm has a keyspace of 139.5-bit, a Normalized Correlation (NC) value of 0.9998 which is closer to 1 and a Peak Signal to Noise Ratio (PSNR) with a value greater than 50 ​dB. Different analyses are also performed on the proposed algorithm using different images. The experimental results show that the proposed scheme is with high key sensitivity and strong robustness against pepper and salt attack and cropping attack. Moreover, the histogram analysis shows that the original carrier image and the final visual image are very similar.
Chain-Structure Echo State Network With Stochastic Optimization: Methodology and Application

In this article, a chain-structure echo state network (CESN) with stacked subnetwork modules is newly proposed as a new kind of deep recurrent neural network for multivariate time series prediction. Motivated by the philosophy of "divide and conquer," the related input vectors are first divided into clusters, and the final output results of CESN are then integrated by successively learning the predicted values of each clustered variable. Network structure, mathematical model, training mechanism, and stability analysis are, respectively, studied for the proposed CESN. In the training stage, least-squares regression is first used to pretrain the output weights in a module-by-module way, and stochastic local search (SLS) is developed to fine-tune network weights toward global optima. The loss function of CESN can be effectively reduced by SLS. To avoid overfitting, the optimization process is stopped when the validation error starts to increase. Finally, SLS-CESN is evaluated in chaos prediction benchmarks and real applications. Four different examples are given to verify the effectiveness and robustness of CESN and SLS-CESN.
Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

We present a relocalization pipeline, which combines an absolute pose regression (APR) network with a novel view synthesis based direct matching module, offering superior accuracy while maintaining low inference time. Our contribution is twofold: i) we design a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering; ii) we show that our method can easily cope with additional unlabeled data without the need for external supervision such as traditional visual odometry or pose graph optimization. As a result, our method achieves state-of-the-art performance among all other single-image APR methods on the 7-Scenes benchmark and the LLFF dataset.
Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding

Visual grounding, i.e., localizing objects in images ac-cording to natural language queries, is an important topic in visual language understanding. The most effective approaches for this task are based on deep learning, which generally require expensive manually labeled image-query or patch-query pairs. To eliminate the heavy depen-dence on human annotations, we present a novel method, named Pseudo-Q, to automatically generate pseudo language queries for supervised training. Our method lever-ages an off-the-shelf object detector to identify visual ob-jects from unlabeled images, and then language queries for these objects are obtained in an unsupervised fashion with a pseudo-query generation module. Then, we design a task-related query prompt module to specifically tailor generated pseudo language queries for visual grounding tasks. Further, in order to fully capture the contextual re-lationships between images and language queries, we de-velop a visual-language model equipped with multi-level cross-modality attention mechanism. Extensive experimen-tal results demonstrate that our method has two notable benefits: (1) it can reduce human annotation costs signifi-cantly, e.g., 31% on Ref Coco [65] without degrading orig-inal model's performance under the fully supervised set-ting, and (2) without bells and whistles, it achieves supe-rior or comparable performance compared to state-of-the-art weakly-supervised visual grounding methods on all the five datasets we have experimented. Code is available at https://github.com/LeapLabTHU/Pseudo-Q.
Physecology: A Conceptual Framework to Describe Data Physicalizations in their Real-World Context

The standard definition for “physicalizations” is “a physical artifact whose geometry or material properties encode data” [ 47 ]. While this working definition provides the fundamental groundwork for conceptualizing physicalization, in practice many physicalization systems go beyond the scope of this definition as they consist of distributed physical and digital elements that involve complex interaction mechanisms. In this article, we examine how “physicalization” is part of a broader ecology—the “physecology”—with properties that go beyond the scope of the working definition. Through analyzing 60 representative physicalization papers, we derived six design dimensions of a physecology: (i) represented data type, (ii) way of information communication, (iii) interaction mechanisms, (iv) spatial input–output coupling, (v) physical setup, and (vi) audiences involved. Our contribution is the extension of the definition of physicalization to the broader concept of “physecology,” to provide conceptual clarity on the design of physicalizations for future work.