Reviewerly/exHarmony huggingface dataset:
Fully Convolutional Neural Networks for Raw Eye Tracking Data Segmentation, Generation, and Reconstruction

In this paper, we use fully convolutional neural networks for the semantic segmentation of eye tracking data. We also use these networks for reconstruction, and in conjunction with a variational auto-encoder to generate eye movement data. The first improvement of our approach is that no input window is necessary, due to the use of fully convolutional networks and therefore any input size can be processed directly. The second improvement is that the used and generated data is raw eye tracking data (position X, Y and time) without preprocessing. This is achieved by pre-initializing the filters in the first layer and by building the input tensor along the z axis. We evaluated our approach on three publicly available datasets and compare the results to the state of the art.
Identifying Key Features from App User Reviews

Due to the rapid growth and strong competition of mobile application (app) market, app developers should not only offer users with attractive new features, but also carefully maintain and improve existing features based on users' feedbacks. User reviews indicate a rich source of information to plan such feature maintenance activities, and it could be of great benefit for developers to evaluate and magnify the contribution of specific features to the overall success of their apps. In this study, we refer to the features that are highly correlated to app ratings as key features, and we present KEFE, a novel approach that leverages app description and user reviews to identify key features of a given app. The application of KEFE especially relies on natural language processing, deep machine learning classifier, and regression analysis technique, which involves three main steps: 1) extracting feature-describing phrases from app description; 2) matching each app feature with its relevant user reviews; and 3) building a regression model to identify features that have significant relationships with app ratings. To train and evaluate KEFE, we collect 200 app descriptions and 1,108,148 user reviews from Chinese Apple App Store. Experimental results demonstrate the effectiveness of KEFE in feature extraction, where an average F-measure of 78.13% is achieved. The key features identified are also likely to provide hints for successful app releases, as for the releases that receive higher app ratings, 70% of features improvements are related to key features.
Universal Neural Vocoding with Parallel Wavenet

We present a universal neural vocoder based on Parallel WaveNet, with an additional conditioning network called Audio Encoder. Our universal vocoder offers real-time high-quality speech synthesis on a wide range of use cases. We tested it on 43 internal speakers of diverse age and gender, speaking 20 languages in 17 unique styles, of which 7 voices and 5 styles were not exposed during training. We show that the proposed universal vocoder significantly outperforms speaker-dependent vocoders overall. We also show that the proposed vocoder outperforms several existing neural vocoder architectures in terms of naturalness and universality. These findings are consistent when we further test on more than 300 open-source voices.
Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparison

We contribute a method to automate parameter configurations for chart layouts by learning from human preferences. Existing charting tools usually determine the layout parameters using predefined heuristics, producing sub-optimal layouts. People can repeatedly adjust multiple parameters (e.g., chart size, gap) to achieve visually appealing layouts. However, this trial-and-error process is unsystematic and time-consuming, without a guarantee of improvement. To address this issue, we develop Layout Quality Quantifier (LQ2), a machine learning model that learns to score chart layouts from paired crowdsourcing data. Combined with optimization techniques, LQ2 recommends layout parameters that improve the charts' layout quality. We apply LQ2 on bar charts and conduct user studies to evaluate its effectiveness by examining the quality of layouts it produces. Results show that LQ2 can generate more visually appealing layouts than both laypeople and baselines. This work demonstrates the feasibility and usages of quantifying human preferences and aesthetics for chart layouts.
Linguistic Steganography Based on Adaptive Probability Distribution

Text has become one of the most extensively used digital media in Internet, which provides steganography an effective carrier to realize confidential message hiding. Nowadays, generation-based linguistic steganography has made a significant breakthrough due to the progress of deep learning. However, previous methods based on recurrent neural network have two deviations including exposure bias and embedding deviation, which seriously destroys the security of steganography. In this article, we propose a novel linguistic steganographic model based on adaptive probability distribution and generative adversarial network, which achieves the goal of hiding secret messages in the generated text while guaranteeing high security performance. First, the steganographic generator is trained by using generative adversarial network to effectively tackle the exposure bias, and then the candidate pool is obtained by a probability similarity function at each time step, which alleviates the embedding deviation through dynamically maintaining the diversity of probability distribution. Third, to further improve the security, a novel strategy that conducts information embedding during model training is put forward. We design various experiments from different aspects to verify the performance of the proposed model, including imperceptibility, statistical distribution, anti-steganalysis ability. demonstrate that our proposed model outperforms the current state-of-the-art steganographic schemes.
Feature Completion for Occluded Person Re-Identification

Person re-identification (reID) plays an important role in computer vision. However, existing methods suffer from performance degradation in occluded scenes. In this work, we propose an occlusion-robust block, Region Feature Completion (RFC), for occluded reID. Different from most previous works that discard the occluded regions, RFC block can recover the semantics of occluded regions in feature space. First, a Spatial RFC (SRFC) module is developed. SRFC exploits the long-range spatial contexts from non-occluded regions to predict the features of occluded regions. The unit-wise prediction task leads to an encoder/decoder architecture, where the region-encoder models the correlation between non-occluded and occluded region, and the region-decoder utilizes the spatial correlation to recover occluded region features. Second, we introduce Temporal RFC (TRFC) module which captures the long-term temporal contexts to refine the prediction of SRFC. RFC block is lightweight, end-to-end trainable and can be easily plugged into existing CNNs to form RFCnet. Extensive experiments are conducted on occluded and commonly holistic reID benchmarks. Our method significantly outperforms existing methods on the occlusion datasets, while remains top even superior performance on holistic datasets. The source code is available at https://github.com/blue-blue272/OccludedReID-RFCnet.
Dynamic Dataflow Scheduling and Computation Mapping Techniques for Efficient Depthwise Separable Convolution Acceleration

Depthwise separable convolution (DSC) has become one of the essential structures for lightweight convolutional neural networks. Nevertheless, its hardware architecture has not received much attention. Several previous hardware designs incur either high off-chip memory traffic or large on-chip memory usage, and hence have deficiency in terms of hardware efficiency as well as performance. This paper proposes two efficient dynamic design techniques, i.e. adaptive row-based dataflow scheduling and adaptive computation mapping, to achieve a much better trade-off between hardware efficiency and performance for DSC-based lightweight CNN accelerator. The effectiveness and efficiency of the proposed dynamic design techniques have been extensively evaluated using six DSC-based lightweight CNNs. Compared with the reference architectures, the simulation results show the proposed architectural techniques can at least reduce on-chip buffer size by 50.4% and improve the performance of convolution calculation by 1.18× while maintaining the minimum off-chip memory traffic. MobileNetV2 is implemented on Zynq UltraScale+ ZCU102 SoC FPGA, and the results show the proposed accelerator can achieve 381.7 frames per second (fps), which is 1.43× of the reference design, and it can save about 36.3% on-chip buffer size compared with the reference design, while maintaining the same off-chip memory traffic.
A Face-Mask Detection Approach based on YOLO Applied for a New Collected Dataset

Since the beginning of the COVID-19 pandemic, many lives are in danger. According to WHO (World Health Organization)'s statements, breathing without a mask is highly dangerous in public and crowded places. Indeed, wearing masks reduces the chance of being infected, and detecting unmasked people is a waste of resources if not performed automatically. AI techniques are used to increase the detection speed of masked and unmasked faces. In this research, a novel dataset and two different methods are proposed to detect masked and unmasked faces in real-time. In the first method, an object detection model is applied to find and classify masked and unmasked faces. In the second method, a YOLO face detector spots faces (whether masked or not), and then the faces are classified into masked and unmasked categories with a novel fast yet effective CNN architecture. By the methods proposed in this paper, the accuracy of 99.5% is achieved on the newly collected dataset.
Holographic Projection Technology in the Field of Digital Media Art

The advent of the digital age has given new forms and new connotations to artistic creation, and more and more digital media technologies have entered the stage of artistic creation and exhibitions. At present, holographic projection technology has become a hot application technology in the field of digital media art. The purpose of this paper is to explore the technical principles of holographic projection technology and its application in the field of digital media art, so as to provide suggestions for the application and promotion of holographic projection technology and the development and innovation of digital media art. First of all, this article understands the technical principles of holographic projection and its application status in various fields, especially in the field of digital media art, through relevant literature research. Then, this article introduces the digital holographic technology, virtual imaging technology, and computer simulation technology used in the realization of holographic projection technology. Then, based on the advantages of holographic projection technology in three-dimensional image recording and reproduction, this paper proposes to introduce holographic projection technology to digital art museums, digital art exhibitions, and other digital media art applications and to study the effect of holographic projection technology on art through simulation experiments, the effect of recording and reproducing the image of the work. Finally, the three-dimensional reconstruction image of the digital holographic projection experiment on the artwork is compared with the simulated image of the Contour GT profiler to verify the feasibility of applying the holographic projection technology to art exhibitions and the effect of three-dimensional image recording and reproduction. Research shows that the holographic projection technology can achieve 93.34% of the simulation effect of recording and reproducing 3D images of artworks. It is also found that 59.86% of the audience who pay attention to the art experience strongly support the application of holographic projection technology in digital media art fields such as digital art gallery. This fully proves the feasibility of applying holographic projection technology to digital art exhibitions and provides a full range of artistic experience for audiences who cannot be present.
Efficient Client Contribution Evaluation for Horizontal Federated Learning

In federated learning (FL), fair and accurate measurement of the contribution of each federated participant is of great significance. The level of contribution not only provides a rational metric for distributing financial benefits among federated participants, but also helps to discover malicious participants that try to poison the FL framework. Previous methods for contribution measurement were based on enumeration over possible combination of federated participants. Their computation costs increase drastically with the number of participants or feature dimensions, making them inapplicable in practical situations. In this paper an efficient method is proposed to evaluate the contributions of federated participants. This paper focuses on the horizontal FL framework, where client servers calculate parameter gradients over their local data, and upload the gradients to the central server. Before aggregating the client gradients, the central server train a data value estimator of the gradients using reinforcement learning techniques. As shown by experimental results, the proposed method consistently outperforms the conventional leave-one-out method in terms of valuation authenticity as well as time complexity.
SNR-Adaptive Deep Joint Source-Channel Coding for Wireless Image Transmission

Considering the problem of joint source-channel coding (JSCC) for multi-user transmission of images over noisy channels, an autoencoder-based novel deep joint source-channel coding scheme is proposed in this paper. In the proposed JSCC scheme, the decoder can estimate the signal-to-noise ratio (SNR) and use it to adaptively decode the transmitted image. Experiments demonstrate that the proposed scheme achieves impressive results in adaptability for different SNRs and is robust to the noise in the SNR estimation of the decoder. To the best of our knowledge, this is the first deep JSCC scheme that focuses on the adaptability for different SNRs and can be applied to multi-user scenarios.
Algorithmic audiencing: Why we need to rethink free speech on social media

Social media platforms, such as Facebook, are today’s agoras, the spaces where public discourse takes place. Freedom of speech on social media has thus become a matter of concern, with calls for better regulation. Public debate revolves around content moderation, seen by some as necessary to remove harmful content, yet as censorship by others. In this article, we argue that the current debate is exclusively focused on the speaking side of speech but overlooks an important way in which platforms have come to interfere with free speech on the audience side. Rather than simply speaking to one’s follower network, algorithms now organise speech on social media with the aim to increase user engagement and marketability for targeted advertising. The result is that audiences for speech are now decided algorithmically, a phenomenon we term ‘algorithmic audiencing’. We put forward algorithmic audiencing as a discovery, a novel phenomenon that has been overlooked so far. We show that it interferes with free speech in unprecedented ways not possible in pre-digital times, by amplifying or suppressing speech for economic gain, which in turn distorts the free and fair exchange of ideas in public discourse. When black-boxed algorithms determine who we speak to the problematic for free speech changes from ‘what can be said’ to ‘what will be heard’ and ‘by whom’. We must urgently problematise the audience side of speech if we want to truly understand, and regulate, free speech on social media. For Information Systems research, algorithmic audiencing opens up entirely new research avenues.
CNN-Based Joint SNR and Doppler Shift Classification Using Spectrogram Images for Adaptive Modulation and Coding

This paper proposes a novel convolutional neural network (CNN) based joint classification method to characterize the signal-to-noise power ratio (SNR) and Doppler shift using spectrogram images, in order to enable efficient adaptive modulation and coding (AMC) designs. It is necessary to maintain high communication performances even in stringent environments where transceivers move at high speed due to the diversification of wireless applications. To optimize the transmission rate in such dynamic environments, AMC scheme is known to be effective. AMC is generally designed based on feedback information (FBI) such as the SNR and Doppler shift acquired on the receiving side. Here, the challenge is an increase in calculation burden, processing delay and estimation accuracy of the FBI. We focused on the spectrogram which is composed of power values in the time and frequency domains. Its two-dimensional fluctuation represents the Doppler shift as well as noise values. In the proposed method, such key information for AMC can be simultaneously extracted from a single spectrogram via a trained CNN. Therefore it is expected to contribute to reducing the computational burden and speeding up the signal processing. Simulation results are presented to demonstrate that the proposed method achieves better performance than traditional methods.
RobotAR: An Augmented Reality Compatible Teleconsulting Robotics Toolkit for Augmented Makerspace Experiences

Distance learning is facing a critical moment finding a balance between high quality education for remote students and engaging them in hands-on learning. This is particularly relevant for project-based classrooms and makerspaces, which typically require extensive trouble-shooting and example demonstrations from instructors. We present RobotAR, a teleconsulting robotics toolkit for creating Augmented Reality (AR) makerspaces. We present the hardware and software for an AR-compatible robot, which behaves as a student's voice assistant and can be embodied by the instructor for teleconsultation. As a desktop-based teleconsulting agent, the instructor has control of the robot's joints and position to better focus on areas of interest inside the workspace. Similarly, the instructor has access to the student's virtual environment and the capability to create AR content to aid the student with problem-solving. We also performed a user study which compares current techniques for distance hands-on learning and an implementation of our toolkit.
Personalization Strategies for End-to-End Speech Recognition Systems

The recognition of personalized content, such as contact names, remains a challenging problem for end-to-end speech recognition systems. In this work, we demonstrate how first- and second-pass rescoring strategies can be leveraged together to improve the recognition of such words. Following previous work, we use a shallow fusion approach to bias towards recognition of personalized content in the first-pass decoding. We show that such an approach can improve personalized content recognition by up to 16% with minimum degradation on the general use case. We describe a fast and scalable algorithm that enables our biasing models to remain at the word-level, while applying the biasing at the subword level. This has the advantage of not requiring the biasing models to be dependent on any subword symbol table. We also describe a novel second-pass de-biasing approach: used in conjunction with a first-pass shallow fusion that optimizes on oracle WER, we can achieve an additional 14% improvement on personalized content recognition, and even improve accuracy for the general use case by up to 2.5%.
Limited Control Over the Body as Intriguing Play Design Resource

Interest in combining interactive play and the human body, using “bodily play” systems, is increasing. While these systems primarily prioritize a player's control over their bodily actions, we see intriguing possibilities in the pursuit of “limited control over the body” as an intriguing design resource for bodily play systems. In this paper, we use three of our bodily play systems to illustrate how designers can engage with limited control over the body by varying the player's degree of indirect control (for instance, via other bodily activity and external triggers). We also propose four strategies for employing limited control over the body: Exploration, Reflection, Learning and Embracement. We hope our own work and the strategies developed from it will assist designers to employ limited control over the body, ultimately helping people benefit from engaging their bodies through play.
“Can I Not Be Suicidal on a Sunday?”: Understanding Technology-Mediated Pathways to Mental Health Support

Individuals in distress adopt varied pathways in pursuit of care that aligns with their individual needs. Prior work has established that the first resource an individual leverages can influence later care and recovery, but less is understood about how the design of a point of care might interact with subsequent pathways to care. We investigate how the design of the Indian mental health helpline system interacts with complex sociocultural factors to marginalize caller needs. We draw on interviews with 18 helpline stakeholders, including individuals who have engaged with helplines in the past, shedding light on how they navigate both technological and structural barriers in pursuit of relief. Finally, we use a design justice framework rooted in Amartya Sen's conceptualization of realization-focused justice to discuss implications and present recommendations towards the design of technology-mediated points of mental health support.
Efficient Compiler Autotuning via Bayesian Optimization

A typical compiler such as GCC supports hundreds of optimizations controlled by compilation flags for improving the runtime performance of the compiled program. Due to the large number of compilation flags and the exponential number of flag combinations, it is impossible for compiler users to manually tune these optimization flags in order to achieve the required runtime performance of the compiled programs. Over the years, many compiler autotuning approaches have been proposed to automatically tune optimization flags, but they still suffer from the efficiency problem due to the huge search space. In this paper, we propose the first Bayesian optimization based approach, called BOCA, for efficient compiler autotuning. In BOCA, we leverage a tree-based model for approximating the objective function in order to make Bayesian optimization scalable to a large number of optimization flags. Moreover, we design a novel searching strategy to improve the efficiency of Bayesian optimization by incorporating the impact of each optimization flag measured by the tree-based model and a decay function to strike a balance between exploitation and exploration. We conduct extensive experiments to investigate the effectiveness of BOCA on two most popular C compilers (i.e., GCC and LLVM) and two widely-used C benchmarks (i.e., cBench and PolyBench). The results show that BOCA significantly outperforms the state-of-the-art compiler autotuning approaches and Bayesion optimization methods in terms of the time spent on achieving specified speedups, demonstrating the effectiveness of BOCA.