Reviewerly/exHarmony huggingface dataset:
Making Everyday Things Talk: Speculative Conversations into the Future of Voice Interfaces at Home

What if things had a voice? What if we could talk directly to things instead of using a mediating voice interface such as an Alexa or a Google Assistant? In this paper, we share our insights from talking to a pair of boots, a tampon, a perfume bottle, and toilet paper among other everyday things to explore their conversational capabilities. We conducted Thing Interviews using a more-than-human design approach to discover a thing's perspectives, worldviews and its relations to other humans and nonhumans. Based on our analysis of the speculative conversations, we identified some themes characterizing the emergent qualities of people's relationships with everyday things. We believe the themes presented in the paper may inspire future research on designing everyday things with conversational capabilities at home.
Improving Prosody Modelling with Cross-Utterance Bert Embeddings for End-to-End Speech Synthesis

Although speech prosody is related to the linguistic information up to the discourse structure, most text-to-speech (TTS) systems only take into account the information within each sentence. This makes it challenging when converting a paragraph of text into natural and expressive speech. In this paper, we propose to use the text embeddings of the neighboring sentences to improve the prosody generation for each utterance of a paragraph in an end-to-end fashion without using any explicit prosody features. More specifically, cross-utterance (CU) context vectors, which are produced by an additional CU encoder based on the sentence embeddings extracted by a pretrained BERT model, are used to augment the input of the Tacotron2 decoder. Two types of BERT embeddings are investigated, which leads to the use of different CU encoder structures. Experimental results on a Mandarin audiobook dataset and the LJ-Speech English audiobook dataset demonstrate the use of CU information can improve the naturalness and expressiveness of the synthesized speech. Subjective listening testing shows most of the participants prefer the voice generated using the CU encoder over that generated using standard Tacotron2. It is also found that the prosody can be controlled indirectly by changing the neighbouring sentences.
Virtual tourism for older adults living in residential care: A mixed-methods study

Due to financial and mobility barriers, a majority of older adults living in collective dwellings are no longer able to engage in tourism, a leisure activity that contributes to quality of life and wellbeing. Immersive Virtual Reality (VR) may serve as a programmatic tool to facilitate tourism. This pilot study examined the effects of VR tourism exposure on indices of psychosocial wellbeing among older adults living in residential care. Using a mixed-methods study design, 18 older adults were exposed to VR tourism three times a week, over six weeks. Participants reported decreased anxiety and fatigue immediately following exposure, and increased social engagement and quality of life following six weeks of VR tourism. Qualitative data offered additional insight on the process by which VR tourism may enhance wellbeing. Findings suggest that immersive VR tourism may be a viable program for older adults in residential care.
PPG-Based Singing Voice Conversion with Adversarial Representation Learning

Singing voice conversion (SVC) aims to convert the voice of one singer to that of other singers while keeping the singing content and melody. On top of recent voice conversion works, we propose a novel model to steadily convert songs while keeping their naturalness and intonation. We build an end-to-end architecture, taking phonetic posteriorgrams (PPGs) as inputs and generating mel spectrograms. Specifically, we implement two separate encoders: one encodes PPGs as content, and the other compresses mel spectrograms to supply acoustic and musical information. To improve the performance on timbre and melody, an adversarial singer confusion module and a mel-regressive representation learning module are designed for the model. Objective and subjective experiments are conducted on our private Chinese singing corpus. Comparing with the baselines, our methods can significantly improve the conversion performance in terms of naturalness, melody, and voice similarity. Moreover, our PPG-based method is proved to be robust for noisy sources.
FaceSight: Enabling Hand-to-Face Gesture Interaction on AR Glasses with a Downward-Facing Camera Vision

We present FaceSight, a computer vision-based hand-to-face gesture sensing technique for AR glasses. FaceSight fixes an infrared camera onto the bridge of AR glasses to provide extra sensing capability of the lower face and hand behaviors. We obtained 21 hand-to-face gestures and demonstrated the potential interaction benefits through five AR applications. We designed and implemented an algorithm pipeline that segments facial regions, detects hand-face contact (f1 score: 98.36%), and trains convolutional neural network (CNN) models to classify the hand-to-face gestures. The input features include gesture recognition, nose deformation estimation, and continuous fingertip movement. Our algorithm achieves classification accuracy of all gestures at 83.06%, proved by the data of 10 users. Due to the compact form factor and rich gestures, we recognize FaceSight as a practical solution to augment input capability of AR glasses in the future.
Unscripted Retargeting: Reach Prediction for Haptic Retargeting in Virtual Reality

Research is exploring novel ways of adding haptics to VR. One popular technique is haptic retargeting, where real and virtual hands are decoupled to enable the reuse of physical props. However, this technique requires the system to know the users' intended interaction target, or requires additional hardware for prediction. We explore software-based reach prediction as a means of facilitating responsive, unscripted retargeting. We trained a Long Short-Term Memory network on users' reach trajectories to predict intended targets. We achieved an accuracy of 81.1 % at approximately 65% of movement. This could enable haptic retargeting during the last 35% of movement. We discuss the implications for possible physical proxy locations.
Reinforcement learning-based QoE-oriented dynamic adaptive streaming framework

Dynamic adaptive streaming over the HTTP (DASH) standard has been widely adopted by many content providers for online video transmission and greatly improve the performance. Designing an efficient DASH system is challenging because of the inherent large fluctuations characterizing both encoded video sequences and network traces. In this paper, a reinforcement learning (RL)-based DASH technique that addresses user quality of experience (QoE) is constructed. The DASH adaptive bitrate (ABR) selection problem is formulated as a Markov decision process (MDP) problem. Accordingly, an RL-based solution is proposed to solve the MDP problem, in which the DASH clients act as the RL agent, and the network variation constitutes the environment. The proposed user QoE is used as the reward by jointly considering the video quality and buffer status. The goal of the RL algorithm is to select a suitable video quality level for each video segment to maximize the total reward. Then, the proposed RL-based ABR algorithm is embedded in the QoE-oriented DASH framework. Experimental results show that the proposed RL-based ABR algorithm outperforms state-of-the-art schemes in terms of both temporal and visual QoE factors by a noticeable margin while guaranteeing application-level fairness when multiple clients share a bottlenecked network.
What Makes a Great Maintainer of Open Source Projects?

Although Open Source Software (OSS) maintainers devote a significant proportion of their work to coding tasks, great maintainers must excel in many other activities beyond coding. Maintainers should care about fostering a community, helping new members to find their place, while also saying "no" to patches that although are well-coded and well-tested, do not contribute to the goal of the project. To perform all these activities masterfully, maintainers should exercise attributes that software engineers (working on closed source projects) do not always need to master. This paper aims to uncover, relate, and prioritize the unique attributes that great OSS maintainers might have. To achieve this goal, we conducted 33 semi-structured interviews with well-experienced maintainers that are the gatekeepers of notable projects such as the Linux Kernel, the Debian operating system, and the GitLab coding platform. After we analyzed the interviews and curated a list of attributes, we created a conceptual framework to explain how these attributes are connected. We then conducted a rating survey with 90 OSS contributors. We noted that "technical excellence" and "communication" are the most recurring attributes. When grouped, these attributes fit into four broad categories: management, social, technical, and personality. While we noted that "sustain a long term vision of the project" and being "extremely careful" seem to form the basis of our framework, we noted through our survey that the communication attribute was perceived as the most essential one.
HDINet: Hierarchical Dual-Sensor Interaction Network for RGBT Tracking

RGBT tracking is a practical solution that combines RGB and thermal infrared modes to solve tracking failures in complex environments to achieve all-day and all-weather work, which makes it gradually applied in multifarious fields. The fundamental reason is that it could avoid the damage of tracking performance caused by the limitation of the imaging characteristics of a single sensor. The existing work aggregates features in different ways, without considering hierarchical complementary interactions and the value of the initial input that may affect subsequent aggregation. In this paper, a novel hierarchical dual-sensor interaction network is proposed, which is mainly composed of dual-sensor interaction, sensor-specific and instance learning. Specifically, our network mainly benefits from the design of two modules, called feature interaction module and data encoding module. The dominant information of the dual sensor is extracted and supplemented by the former based on attention. The latter encodes the raw data into the initial input of the first feature interaction module, whose quality has a key influence on the follow-up. We investigate the performance through extensive experiments compared with the recent state-of-the-art RGB and RGBT trackers on the GTOT and RGBT234 datasets, which verify that our network is effective in both quantitative and qualitative evaluation.
Block Convolution: Toward Memory-Efficient Inference of Large-Scale CNNs on FPGA

Deep convolutional neural networks have achieved remarkable progress in recent years. However, the large volume of intermediate results generated during inference poses a significant challenge to the accelerator design for resource-constrained field-programmable gate array (FPGA). Due to the limited on-chip storage, partial results of intermediate layers are frequently transferred back and forth between on-chip memory and off-chip DRAM, leading to a nonnegligible increase in latency and energy consumption. In this article, we propose block convolution, a hardware-friendly, simple, yet efficient convolution operation that can completely avoid the off-chip transfer of intermediate feature maps at runtime. The fundamental idea of block convolution is to eliminate the dependency of feature map tiles in the spatial dimension when spatial tiling is used, which is realized by splitting a feature map into independent blocks so that convolution can be performed separately on individual blocks. We conduct extensive experiments to demonstrate the efficacy of the proposed block convolution on both the algorithm side and the hardware side. Specifically, we evaluate block convolution on: 1) VGG-16, ResNet-18, ResNet-50, and MobileNet-V1 for the ImageNet classification task; 2) SSD and FPN for the COCO object detection task; and 3) VDSR for the Set5 single-image superresolution task. Experimental results demonstrate that comparable or higher accuracy can be achieved with block convolution. We also showcase two CNN accelerators via algorithm/hardware co-design based on block convolution on memory-limited FPGAs, and evaluation shows that both accelerators substantially outperform the baseline without off-chip transfer of intermediate feature maps.
Reinforced Anchor Knowledge Graph Generation for News Recommendation Reasoning

News recommendation systems play a key role in online news reading service. Knowledge graphs (KG), which contain comprehensive structural knowledge, are well known for their potential to enhance both accuracy and explainability. While existing works intensively study using KG to improve news recommendation accuracy, using KG for news recommendation reasoning has not been fully explored. A few works such as KPRN [18], [22] and ADAC [25] have discussed knowledge reasoning in some other recommendation domains such as music or movie, but their methods are not practical for the news. How to make reasoning scalable to generic KGs, easy to deploy for real-time serving and meanwhile elastic for both recall and ranking stages remains an open question.
2D Logistic-Adjusted-Chebyshev map for visual color image encryption

With the increasing requirements of the performance in secure communication filed based on chaos, chaotic systems have been gradually developed into higher dimensions. Because the sequence generated by one-dimension chaos, such as Chebyshev map has the problem of weak chaotic properties and uneven distribution, a new two-dimensional chaotic map, called Logistic-Adjusted-Chebyshev map (2D-LACM), is proposed in this paper which enlarges the range of chaotic control parameter, and expands the range of surjection to the entire control parameter interval to make the distribution of the sequence more uniform. Compared with other improved chaotic maps, the proposed 2D-LACM in this paper has more complex chaotic properties and better ergodicity. Next, a visualized image encryption algorithm based on 2D-LACM is proposed (LACM-VIEA), which can encrypt the plain image into a noisy image, and then embed it into a cover image of the same size. In order to demonstrate the security of LACM-VIEA, the algorithm is analyzed in detail from the aspects of key, histogram, visual effect, robustness and anti-plaintext attacks, etc. The final analysis results show that LACM-VIEA has good security performance.
HIDRA: A Distributed Blockchain-Based Architecture for Fog/Edge Computing Environments

Nowadays, the fog computing paradigm is being consolidated as a solution for processing the explosion of data generated by lots of common IoT devices connected to the Internet. In contrast to cloud computing, fog computing achieves efficient data processing without incurring large latencies or data transfers to/from the cloud. The distributed nature of fog computing makes the resource orchestration a challenge. Commonly used centralized solutions are of limited utility in this context. Moreover, fog nodes are usually resource constrained, so the implementation of the management modules should be carefully designed in order not to cause excessive overheads. On the other hand, blockchain has proven its utility beyond cryptocurrencies, to support distributed and reliable information storage. When combined with smart contracts it can provide a distributed computer where all the nodes independently and equally contribute to a common global system state, which must be agreed by consensus. This also provides inherent desirable features, such as immutability and transparency. In this work, a novel architecture called HIDRA is presented, aimed at resource orchestration in fog computing environments based on a Ethereum blockchain implementation. A prototype implementation on a testbed composed of single-board computers has been carried out. Results show the low overhead introduced into the system and its ability to perform a coordinated action among fog nodes without the intervention of any central authority.
Ring-Regularized Cosine Similarity Learning for Fine-Grained Face Verification

Face verification aims to determine whether a pair of face images belong to the same person. Different from the traditional face verification, the negative sample pairs in fine-grained face verification are composed of similar face images, e.g., facial images of twins, which makes it still very challenging. In this paper, we investigate the fine-grained face verification problem via metric learning techniques, and propose a ring-regularized cosine similarity learning (RRCSL) method to distinguish the negative face pairs. The proposed RRCSL method seeks a linear transformation to enlarge the cosine similarity of intra-class and reduce the cosine similarity of inter-class as much as possible, and adaptively learns the norm of samples to the scaled circle by exploiting the ring regularization term simultaneously. Experimental results on three face datasets demonstrate the effectiveness of RRCSL for fine-grained face verification.
Stepping ahead Firefly Algorithm and hybridization with evolution strategy for global optimization problems

Intelligent optimization algorithms based on swarm principles have been widely researched in recent times. The Firefly Algorithm (FA) is an intelligent swarm algorithm for global optimization problems. In literature, FA has been seen as an efficient and robust optimization algorithm. FA is an algorithm that has obtained good to best results in complex problems. Therefore, there are many instances of modification, and hybridization of FA with other optimizing algorithms, but further improvements are still possible. This research first proposes a new modification of FA by introducing a novel and unique stepping ahead parameter. The concept is based on being proactive rather than reactive, which is the normal behavior of a standard FA. The notion behind stepping ahead is to send a firefly ahead then the best known position to look for even better solution. Second, a new design of a hybrid of the newly modified FA with Covariance Matrix Adaptation Evolution Strategy (CMAES) to improve the exploitation further while maintaining good exploration in the fireflies is presented. The main use of CMAES in this hybrid algorithm is to provide diversity to fireflies, as a result it improves exploitation. Traditionally, hybridization has combined two or more algorithms in terms of structure only, and consideration was not given to the increase in time complexity or diversity. In this paper, the two algorithms are not run in separate cycles rather CMAES is placed inside the FA. Through this novelty, CMAES is initiated inside FA loop and an extra loop is avoided and at the same time CMAES diversifies FA solutions. The structure of algorithm together with the strength of individual solution are used. The newly established modified FA and hybrid are used to solve selected sixty five global optimization benchmark problems together with eight real-world problems from CEC 2011. The proposed algorithms have outperformed FA algorithm in both benchmark and real-world problems. The optimal solutions found by FA in benchmark problems was 69.2% while FA-Step algorithm achieved 73.9% and FA-CMAES algorithm obtained 92.3%. In real-world problem, FA obtained 37.5%, FA-Step was 50% while FA-CMAES was 75%. The results invariably show that the proposed algorithms perform significantly better than the standalone methods as well as the algorithms from the literature.
Dual attention guided multi-scale CNN for fine-grained image classification

For the classification of fine-grained images, the subtle differences among the subclasses of the main category must be distinguished. Intuitively, the key to realizing the fine-grained image categorization lies in locating and identifying the detailed differences in the local regions and capturing their feature representations. In this paper, we propose utilizing an attention module combined with a multi-scale latent representation network to locate the discriminative spatial regions, and then learn an accurate attention map to assist the category decision. Furthermore, an attention module is also employed to determine the channel weights of the distinct scale feature maps before the final step. Extensive experiments demonstrate that our model obtains a competitive performance against state-of-the-art baselines on two benchmark datasets, the attention validation experiments further reveal the ability of the model in choosing the proper channel features for low-quality image categorization.
Non-interactive Zero Knowledge from Sub-exponential DDH

We provide the first constructions of non-interactive zero-knowledge and Zap arguments for NP based on the sub-exponential hardness of Decisional Diffie-Hellman against polynomial time adversaries (without use of groups with pairings). Central to our results, and of independent interest, is a new notion of interactive trapdoor hashing protocols.
Normalized edge convolutional networks for skeleton-based hand gesture recognition

Dynamic hand skeletons consisting of discrete spatial-temporal finger joint clouds effectively convey the intentions of communicators. Previous graph convolutional networks (GCNs) relying on human hand-crafted inductive biases have been quickly promoted for skeleton-based hand gesture recognition (SHGR). However, most existing graph constructions for GCN-based solutions are set manually, only considering the physical topology of the hand skeleton, and the fixed dependencies among hand joints may lead to suboptimal models. To enrich the local dependencies, we emphasize that hand skeletons can be seen from two views: explicit joint clouds and implicit skeleton topology. Starting from those two views of hand gestures, we attempt to introduce dynamics and diversities into the local neighborhood of the graph by dividing it into sets of physical neighbors, temporal neighbors and varying neighbors. Next, we systematically proceed with three innovations, including the novel edge-varying graph, normalized edge convolution operation, and zig-zag sampling strategy, to alleviate the challenges resulting from engineering practices. Finally, spatial-based GCNs called normalized edge convolutional networks are constructed for hand gesture recognition. Experiments on publicly available hand datasets show that our work is stable for performing state-of-the-art gesture recognition, and ablation experiments are also provided to validate each contribution.
Individual privacy in data mining using fuzzy optimization

This article proposes the individual data privacy during collaborative computation in data mining method using an optimization model. The privacy problem is solved using different methodologies. The solution for individual privacy is considered as a multi-objective optimization model. Practically, the requirement for privacy varies from user to user. Therefore, it generates inherent vagueness for individual privacy. In this article, the vagueness is considered and the privacy problem is solved by a fuzzy optimization method. The fuzzy multi-objective optimization model is proposed to be used as a supplementary privacy method to address individual privacy issues. The fuzzy constraints are generated to solve the models on the basis of the privacy requirements of users. The fuzzy set domain for the optimization problem is used to fulfil the individual privacy requirements in a computing environment. The proposed solution allows data owners to choose their own privacy level on demand, with maximum flexibility.