Reviewerly/exHarmony huggingface dataset:
Modality Dropout for Improved Performance-driven Talking Faces

We describe our novel deep learning approach for driving animated faces using both acoustic and visual information. In particular, speech-related facial movements are generated using audiovisual information, and non-verbal facial movements are generated using only visual information. To ensure that our model exploits both modalities during training, batches are generated that contain audio-only, video-only, and audiovisual input features. The probability of dropping a modality allows control over the degree to which the model exploits audio and visual information during training. Our trained model runs in real-time on resource limited hardware (e.g. a smart phone), it is user agnostic, and it is not dependent on a potentially error-prone transcription of the speech. We use subjective testing to demonstrate: 1) the improvement of audiovisual-driven animation over the equivalent video-only approach, and 2) the improvement in the animation of speech-related facial movements after introducing modality dropout. Without modality dropout, viewers prefer audiovisual-driven animation in 51% of the test sequences compared with only 18% for video-driven. After introducing dropout viewer preference for audiovisual-driven animation increases to 74%, but decreases to 8% for video-only.
Semantic Image Segmentation on Snow Driving Scenarios

The challenges of driving on snow and ice roads bring out a demand for object detection and drivable area segmentation in snowy environments. Semantic segmentation techniques have been able to achieve good results provided that the models are well-trained based on the appropriate dataset. However, no current driving dataset exists that contains adequate images in snowy environments. To address this issue, we introduce our snowy driving dataset to train and test models for pixel-wise semantic labeling. This snowy driving dataset consists of both real and synthetic samples with 11 classes. We conduct comparative experiments based on a series of the proposed dataset, as well as provide statistics and visual results to show improvement.
An Empirical Study on the Design and Evolution of NoSQL Database Schemas

We study how software engineers design and evolve their domain model when building applications against NoSQL data stores. Specifically, we target Java projects that use object-NoSQL mappers to interface with schema-free NoSQL data stores. This is a popular software stack. Given the source code of ten real-world database applications, we extract the NoSQL database schema thus implied in the application code. We can confirm that schemas are generally denormalized, as is recommended practice in data modeling for NoSQL data stores. Further, we analyze the entire project history, and with it, the evolution history of the NoSQL database schema. We show that NoSQL schemas evolve in all analyzed projects, and observe a comparatively high frequency in schema changes. In doing so, we conduct the so far largest empirical study on NoSQL schema design and evolution.
Unsupervised Recognition of Multi-Resident Activities in Smart-Homes

Several methods have been proposed in the last two decades to recognize human activities based on sensor data acquired in smart-homes. While most existing methods assume the presence of a single inhabitant, a few techniques tackle the challenging issue of multi-resident activity recognition. To the best of our knowledge, all existing methods for multi-inhabitant activity recognition require the acquisition of a labeled training set of activities and sensor events. Unfortunately, activity labeling is costly and may disrupt the users' privacy. In this article, we introduce a novel technique to recognize multi-inhabitant activities without the need of labeled datasets. Our technique relies on an unlabeled sensor data stream acquired from a single resident, and on ontological reasoning to extract probabilistic associations among sensor events and activities. Extensive experiments with a large dataset of multi-inhabitant activities show that our technique achieves an average accuracy very close to the one of state-of-the-art supervised methods, without requiring the acquisition of labeled data.
More Efficient MPC from Improved Triple Generation and Authenticated Garbling

Recent works on distributed garbling have provided highly efficient solutions for constant-round MPC tolerating an arbitrary number of corruptions. In this work, we improve upon state-of-the-art protocols in this paradigm for further performance gain. First, we propose a new protocol for generating authenticated AND triples, which is a key building block in many recent works. \beginitemize \item We propose a new authenticated bit protocol in the two-party and multi-party settings from bare IKNP OT extension, allowing us to reduce the communication by about $24%$ and eliminate many computation bottlenecks. We further improve the computational efficiency for multi-party authenticated AND triples with cheaper and fewer consistency checks and fewer hash function calls. \item We implemented our triple generation protocol and observe around $4\times$ to $5\times$ improvement compared to the best prior protocol in most settings. For example, in the two-party setting with 10 Gbps network and 8 threads, our protocol can generate more than $4$ million authenticated triples per second, while the best prior implementation can only generate $0.8$ million triples per second. In the multi-party setting, our protocol can generate more than $37000$ triples per second over 80 parties, while the best prior protocol can only generate the same number of triples per second over 16 parties. \enditemize We also improve the state-of-the-art multi-party authenticated garbling protocol. \beginitemize \item We take the first step towards applying half-gates in the multi-party setting, which enables us to reduce the size of garbled tables by $2κ$ bits per gate per garbler, where κ is the computational security parameter. This optimization is also applicable in the semi-honest multi-party setting. \item We further reduce the communication of circuit authentication from $4ρ$ bits to $1$ bit per gate, using a new multi-party batched circuit authentication, where ρ is the statistical security parameter. Prior solution with similar efficiency is only applicable in the two-party setting. \enditemize For example, in the three-party setting, our techniques can lead to roughly a $35%$ reduction in the size of a distributed garbled circuit.
Design and evaluation of an integrated scanning laser triangulation sensor

This work presents the design, simulation, control and experimental result of an optical scanning laser sensor for 3D imaging. The system design satisfies the Scheimpflug condition even though only the illumination path is scanned by a compact fast steering mirror (FSM). To reconstruct the surface profile from measured quantities and model parameters, geometrical relations are used. The influence of a misalignment on the achievable resolution is determined, using ray-tracing simulations. To scan the area of interest an adapted scan trajectory is employed, such that a uniform spatial resolution is generated. The system captures the sample with 100 x 100 pixels and achieves a frametime of 10 s. The lateral scan area of 15 x 23 mm is obtained with a maximum lateral resolution of 60 µm. Through the optical scan the measurement time can be reduced by a factor of 400 compared to a mechanical scanning triangulation sensor system.
Text Captcha Is Dead? A Large Scale Deployment and Empirical Study

The development of deep learning techniques has significantly increased the ability of computers to recognize CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart), thus breaking or mitigating the security of existing captcha schemes. To protect against these attacks, recent works have been proposed to leverage adversarial machine learning to perturb captcha pictures. However, they either require the prior knowledge of captcha solving models or lack adaptivity to the evolving behaviors of attackers. Most importantly, none of them has been deployed in practical applications, and their practical applicability and effectiveness are unknown.
End-to-end Dynamic Matching Network for Multi-view Multi-person 3D Pose Estimation

As an important computer vision task, 3d human pose estimation in a multi-camera, multi-person setting has received widespread attention and many interesting applications have been derived from it. Traditional approaches use a 3d pictorial structure model to handle this task. However, these models suffer from high computation costs and result in low accuracy in joint detection. Recently, especially since the introduction of Deep Neural Networks, one popular approach is to build a pipeline that involves three separate steps: (1) 2d skeleton detection in each camera view, (2) identification of matched 2d skeletons and (3) estimation of the 3d poses. Many existing works operate by feeding the 2d images and camera parameters through the three modules in a cascade fashion. However, all three operations can be highly correlated. For example, the 3d generation results may affect the results of detection in step 1, as does the matching algorithm in step 2. To address this phenomenon, we propose a novel end-to-end training scheme that brings the three separate modules into a single model. However, one outstanding problem of doing so is that the matching algorithm in step 2 appears to disjoint the pipeline. Therefore, we take our inspiration from the recent success in Capsule Networks, in which its Dynamic Routing step is also disjointed, but plays a crucial role in deciding how gradients are flowed from the upper to the lower layers. Similarly, a dynamic matching module in our work also decides the paths in which gradients flow from step 3 to step 1. Furthermore, as a large number of cameras are present, the existing matching algorithm either fails to deliver a robust performance or can be very inefficient. Thus, we additionally propose a novel matching algorithm that can match 2d poses from multiple views efficiently. The algorithm is robust and able to deal with situations of incomplete and false 2d detection as well.
Did You Notice the Ads? Examining the Influence of Telepresence and User Control on the Effectiveness of Embedded Billboard Ads in a VR Racing Game

Abstract Virtual reality (VR) can enhance game players’ experience by creating a hyperrealistic environment through telepresence. While marketers have found opportunities for advertising in this new medium, little is known about how consumers process displayed ads in VR. This study examines the impact of telepresence and active control on the effectiveness of embedded in-game ads in a custom-built car racing game. A total of 88 undergraduate students participated in a 2 (telepresence: VR versus flat screen) × 2 (active control: playing versus viewing) between-subject experiment. Brand recall, brand recognition, product purchase intention, and advertised product price estimates were measured. The results showed that billboard ads were the least likely to be remembered by players and the most likely to be remembered by viewers. We did not find a difference in brand recall and recognition between the VR and flat-screen conditions. However, regardless of game control, VR led to higher price estimates. We also found an interaction effect of telepresence and control on purchase intention.
A fully CMOS true random number generator based on hidden attractor hyperchaotic system

Abstract Low-power devices used in Internet-of-things networks have been short of security due to the high power consumption of random number generators. This paper presents a low-power hyperchaos-based true random number generator, which is highly recommended for secure communications. The proposed system, which is based on a four-dimensional chaotic system with hidden attractors and oscillators, exhibits rich dynamics. Numerical analysis is provided to verify the dynamic characteristics of the proposed system. A fully customized circuit is deployed using 130 nm CMOS technology to enable integration into low-power devices. Four output signals are used to seed a SHIFT-XOR-based chaotic data post-processing to generate random bit output. The chip prototype was simulated and tested at 100 MHz sampling frequency. The hyperchaotic circuit consumes a maximum of 980 $$\upmu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>μ</mml:mi></mml:math> W in generating chaotic signals while dissipates a static current of 623 $$\upmu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>μ</mml:mi></mml:math> A. Moreover, the proposed system provides ready-to-use binary random bit sequences which have passed the well-known statistical randomness test suite NIST SP800-22. The proposed novel system design and its circuit implementation provide a best energy efficiency of 4.37 pJ/b at a maximum sampling frequency of 100 MHz.
Static source code metrics and static analysis warnings for fine-grained just-in-time defect prediction

Software quality evolution and predictive models to support decisions about resource distribution in software quality assurance tasks are an important part of software engineering research. Recently, a fine-grained just-in-time defect prediction approach was proposed which has the ability to find bug-inducing files within changes instead of only complete changes. In this work, we utilize this approach and improve it in multiple places: data collection, labeling and features. We include manually validated issue types, an improved SZZ algorithm which discards comments, whitespaces and refactorings. Additionally, we include static source code metrics as well as static analysis warnings and warning density derived metrics as features. To assess whether we can save cost we incorporate a specialized defect prediction cost model. To evaluate our proposed improvements of the fine-grained just-in-time defect prediction approach we conduct a case study that encompasses 38 Java projects, 492,241 file changes in 73,598 commits and spans 15 years. We find that static source code metrics and static analysis warnings are correlated with bugs and that they can improve the quality and cost saving potential of just-in-time defect prediction models.
PRDNet: Medical image segmentation based on parallel residual and dilated network

This paper presents an improved semantic segmentation model for medical images named PRDNet. Common semantic segmentation algorithms only feed the features of the last layer of convolutional neural network into the decoder, which results in the position information of the first several layers of convolutional neural network is not utilized. While in our work, ResNet and dilated convolution are simultaneously used to extract multi-layer features of medical images in parallel. In the decoding stage, the multi-layer features are fused according to the structure of feature pyramid network. Moreover, several classic semantic segmentation algorithms were selected for comparison, such as UNet, Attention UNet, FPN, Deeplab v3, SENet and DANet. After a series of experiments on CHAOS and ISIC2017 datasets, the algorithm proposed by us has a 1%–4% improvement in different evaluation metrics compared with other algorithms.
Federated Mimic Learning for Privacy Preserving Intrusion Detection

Internet of things (IoT) devices are prone to attacks due to the limitation of their privacy and security components. These attacks vary from exploiting backdoors to disrupting the communication network of the devices. Intrusion Detection Systems (IDS) play an essential role in ensuring information privacy and security of IoT devices against these attacks. Recently, deep learning-based IDS techniques are becoming more prominent due to their high classification accuracy. However, conventional deep learning techniques jeopardize user privacy due to the transfer of user data to a centralized server. Federated learning (FL) is a popular privacy-preserving decentralized learning method. FL enables training models locally at the edge devices and transferring local models to a centralized server instead of transferring sensitive data. Nevertheless, FL can suffer from reverse engineering ML attacks that can learn information about the user's data from model. To overcome the problem of reverse engineering, mimic learning is another way to preserve the privacy of ML-based IDS. In mimic learning, a student model is trained with the public dataset, which is labeled with the teacher model that is trained by sensitive user data. In this work, we propose a novel approach that combines the advantages of FL and mimic learning, namely federated mimic learning to create a distributed IDS while minimizing the risk of jeopardizing users' privacy, and benchmark its performance compared to other ML-based IDS techniques using NSL-KDD dataset. Our results show that we can achieve 98.11% detection accuracy with federated mimic learning.
Joint Versus Independent Multiview Hashing for Cross-View Retrieval

Thanks to the low storage cost and high query speed, cross-view hashing (CVH) has been successfully used for similarity search in multimedia retrieval. However, most existing CVH methods use all views to learn a common Hamming space, thus making it difficult to handle the data with increasing views or a large number of views. To overcome these difficulties, we propose a decoupled CVH network (DCHN) approach which consists of a semantic hashing autoencoder module (SHAM) and multiple multiview hashing networks (MHNs). To be specific, SHAM adopts a hashing encoder and decoder to learn a discriminative Hamming space using either a few labels or the number of classes, that is, the so-called flexible inputs. After that, MHN independently projects all samples into the discriminative Hamming space that is treated as an alternative ground truth. In brief, the Hamming space is learned from the semantic space induced from the flexible inputs, which is further used to guide view-specific hashing in an independent fashion. Thanks to such an independent/decoupled paradigm, our method could enjoy high computational efficiency and the capacity of handling the increasing number of views by only using a few labels or the number of classes. For a newly coming view, we only need to add a view-specific network into our model and avoid retraining the entire model using the new and previous views. Extensive experiments are carried out on five widely used multiview databases compared with 15 state-of-the-art approaches. The results show that the proposed independent hashing paradigm is superior to the common joint ones while enjoying high efficiency and the capacity of handling newly coming views.
Amplitude and Frequency Modulation-based features for detection of replay Spoof Speech

Replay attack poses a great threat to the Automatic Speaker Verification (ASV) system. This paper introduces Amplitude Modulation and Frequency Modulation-based features for replay Spoof Speech Detection (SSD) task. In this context, we propose Instantaneous Amplitude (IA) and Instantaneous Frequency (IF) features using Energy Separation Algorithm (ESA). The speech signal is passed through bandpass (subband) filters to obtain narrowband components because speech is a combination of several monocomponent signals. To obtain a narrowband filtered signal, we have used linearly-spaced Butterworth and Gabor filterbanks. The instantaneous modulations helps to understand the local characteristics of a non-stationary signal. These IA and IF components are able to capture the information present in a slowly-varying amplitude envelope and fast-varying frequency. The slow-varying temporal modulations for replay speech have the distorted amplitude envelope, and the fast-varying temporal modulation do not preserve the harmonic structure compared to the natural speech signal. For replay speech signal, the intermediate device characteristics and acoustic environment distorts the spectral energy compared to the natural speech energy. Experiments were performed on the ASVspoof 2017 challenge version 2.0 database with Gaussian Mixture Model (GMM) as a classifier. When ESA-IACC and ESA-IFCC feature sets are fused with Constant Q Cepstral Coefficients (CQCC) feature set at the score-level, the % EER further reduces to 11.93% and 10.12%, respectively, on the evaluation set. In addition, for evaluation set, we have also studied the performance of proposed feature sets on different Replay Configurations (RC), namely, acoustic environments, playback, and recording devices. For all the levels of threat conditions (i.e., low, medium, and high-level) to the ASV system, the proposed feature sets performed better compared to the existing state-of-the-art feature sets. In addition to the ASVspoof 2017 Challenge database, we also performed experiments on other spoofing databases, namely, BTAS 2016, ASVspoof 2019 Challenge database, and Real PA of ASVspoof 2019 Challenge database. For all the spoofing databases used in this study, the proposed ESA-based feature sets perform significantly better than the other feature sets.
Controls on soil geochemistry fractal characteristics in Lemesos (Limassol), Cyprus

This study extends the program of urban soil geochemical mapping in Cyprus to Lemesos, the country's second largest and fastest developing urban centre. Data from 411 soil samples are combined with samples previously analysed as part of the detailed soil geochemical atlas of Cyprus (GAC). Parent lithology is the dominant control on the distribution of most elements with soils derived from carbonate units contrasting with those developed on alluvium-colluvium that contains input from both the carbonates and mafic-ultramafic units of the nearby Troodos Ophiolite. Anthropogenic contamination is best indicated by the Pb-Sn-Ag-Bi suite with elevated values in a belt crossing the southern and older section of the city that is unrelated to parent lithology or current land use classification. As in Lefkosia, children's playgrounds display much higher As contents than at other sites due to application of a natural secondary carbonate ('pouri') as the surface layer, and elevated concentrations of Cu, Fe, Ti, Al, S and Mn associated with basaltic soils used in most school grounds. Multivariate patterns in the Lemesos data largely reflect those of similar parent lithologies within the national soil atlas, but there is closer clustering of the contamination-related elements and an ultramafic suite involving Ni-Cr-Pt-Pd-V-Co in Lemesos. Whereas most elements display multifractal populations using number-size fractal modelling, in urban and contaminated areas some higher concentration element populations do not display fractal behaviour, thus providing a means of identifying anthropogenic influences.
Blurring boundaries between humans and technology: postdigital, postphenomenology and actor-network theory in qualitative research

(2021). Blurring boundaries between humans and technology: postdigital, postphenomenology and actor-network theory in qualitative research. Qualitative Research in Sport, Exercise and Health: Vol. 13, Contemporary digital qualitative research in sport, exercise and health, pp. 26-40.