Reviewerly/exHarmony huggingface dataset:
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning

Previous works have shown that automatic speaker verification (ASV) is seriously vulnerable to malicious spoofing attacks, such as replay, synthetic speech, and recently emerged adversarial attacks. Great efforts have been dedicated to defending ASV against replay and synthetic speech; however, only a few approaches have been explored to deal with adversarial attacks. All the existing approaches to tackle adversarial attacks for ASV require the knowledge for adversarial samples generation, but it is impractical for defenders to know the exact attack algorithms that are applied by the in-the-wild attackers. This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. Inspired by self-supervised learning models (SSLMs) that possess the merits of alleviating the superficial noise in the inputs and reconstructing clean samples from the interrupted ones, this work regards adversarial perturbations as one kind of noise and conducts adversarial defense for ASV by SSLMs. Specifically, we propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. The purification module aims at alleviating the adversarial perturbations in the samples and pulling the contaminated adversarial inputs back towards the decision boundary. Experimental results show that our proposed purification module effectively counters adversarial attacks and outperforms traditional filters from both alleviating the adversarial noise and maintaining the performance of genuine samples. The detection module aims at detecting adversarial samples from genuine ones based on the statistical properties of ASV scores derived by a unique ASV integrating with different number of SSLMs. Experimental results show that our detection module helps shield the ASV by detecting adversarial samples. Both purification and detection methods are helpful for defending against different kinds of attack algorithms. Moreover, since there is no common metric for evaluating the ASV performance under adversarial attacks, this work also formalizes evaluation metrics for adversarial defense considering both purification and detection based approaches into account. We sincerely encourage future works to benchmark their approaches based on the proposed evaluation framework.
A Novel Cloud-Assisted Secure Deep Feature Classification Framework for Cancer Histopathology Images

The advancements in the Internet of Things (IoT) and cloud services have enabled the availability of smart e-healthcare services in a distant and distributed environment. However, this has also raised major privacy and efficiency concerns that need to be addressed. While sharing clinical data across the cloud that often consists of sensitive patient-related information, privacy is a major challenge. Adequate protection of patients’ privacy helps to increase public trust in medical research. Additionally, DL-based models are complex, and in a cloud-based approach, efficient data processing in such models is complicated. To address these challenges, we propose an efficient and secure cancer diagnostic framework for histopathological image classification by utilizing both differential privacy and secure multi-party computation. For efficient computation, instead of performing the whole operation on the cloud, we decouple the layers into two modules: one for feature extraction using the VGGNet <?TeX $_{16}$?> module at the user side and the remaining layers for private prediction over the cloud. The efficacy of the framework is validated on two datasets composed of histopathological images of the canine mammary tumor and human breast cancer. The application of differential privacy preserving to the proposed model makes the model secure and capable of preserving the privacy of sensitive data from any adversary, without significantly compromising the model accuracy. Extensive experiments show that the proposed model efficiently achieves the trade-off between privacy and model performance.
Scalable teacher forcing network for semi-supervised large scale data streams

The large-scale data stream problem refers to high-speed information flow which cannot be processed in scalable manner under a traditional computing platform. This problem also imposes expensive labelling cost making the deployment of fully supervised algorithms unfeasible. On the other hand, the problem of semi-supervised large-scale data streams is little explored in the literature because most works are designed in the traditional single-node computing environments while also being fully supervised approaches. This paper offers Weakly Supervised Scalable Teacher Forcing Network (WeScatterNet) to cope with the scarcity of labelled samples and the large-scale data streams simultaneously. WeScatterNet is crafted under distributed computing platform of Apache Spark with a data-free model fusion strategy for model compression after parallel computing stage. It features an open network structure to address the global and local drift problems while integrating a data augmentation, annotation and auto-correction (DA3) method for handling partially labelled data streams. The performance of WeScatterNet is numerically evaluated in the six large-scale data stream problems with only 25% label proportions. It shows highly competitive performance even if compared with fully supervised learners with 100% label proportions.
Federated Learning with Personalized Local Differential Privacy

Recently, federated learning (FL), as an advanced and practical solution, has been applied to deal with privacy-preserving issues in distributed multi-party federated modeling. However, most existing FL methods focus on the same privacy-preserving budget while ignoring various privacy requirements of participants. In this paper, we for the first time propose an algorithm (PLU-FedOA) to optimize the deep neural network of horizontal FL with personalized local differential privacy. For such considerations, we design two approaches: PLU, which allows clients to upload local updates under differential privacy-preserving of personally selected privacy level, and FedOA, which helps the server aggregates local parameters with optimized weight in mixed privacy-preserving scenarios. Moreover, we theoretically analyze the effect on privacy and optimization of our approaches. Finally, we verify PLU-FedOA on real-world datasets.
A Stealth Program Injection Attack against S7-300 PLCs

Industrial control systems (ICSs) consist of programmable logic controllers (PLCs) which communicate with an engineering station on one side, and control a certain physical process on the other side. Siemens PLCs, particularly S7-300 controllers, are widely used in industrial systems, and modern critical infrastructures heavily rely on them. But unfortunately, security features are largely absent in such devices or ignored/disabled because security is often at odds with operations. As a consequence of the already reported vulnerabilities, it is possible to leverage PLCs and perhaps even the corporate IT network. In this paper we show that S7-300 PLCs are vulnerable and demonstrate that exploiting the execution process of the logic program running in a PLC is feasible. We discuss a replay attack that compromises the password protected PLCs, then we show how to retrieve the Bytecode from the target and decompile the Bytecode to STL source code. Afterwards we present how to conduct a typical injection attack showing that even a very tiny modification in the code is sufficient to harm the target system. Finally we combine the replay attack with the injection approach to achieve a stronger attack - the stealth program injection attack - which can hide the previous modification by engaging a fake PLC, impersonating the real infected device. For real scenarios, we implemented all our attacks on a real industrial setting using S7-300 PLC. We eventually suggest mitigation approaches to secure systems against such threats.
Privacy-Preserving Matrix Factorization for Cross-Domain Recommendation

Cross-domain recommender systems are known to provide solutions to the cold start and data sparsity problems in recommender systems. This can be achieved by leveraging sufficient ratings and users' profiles in one domain to enhance accurate recommendations in another domain. However, domains with sufficient ratings are not willing to share their users' ratings with other recommender systems or domains due to users' privacy and legal concern. Hence this shows a need for a privacy-preserving mechanism that encourages secure knowledge transfer between different domains. This study proposes a privacy-preserving cross-domain recommender system based on matrix factorization. Specifically, the study formally described the privacy requirements of a cross-domain recommender system, which are different from a single domain recommender system. It designs a new framework for a privacy-preserving cross-domain recommender system and then utilized the somewhat homomorphic encryption (SWHE) scheme to ensure users' privacy. The SWHE scheme was used to encrypt users' ratings in different domains, shared latent factor approach was implemented between the domains and extracted knowledge was securely transferred from the source domain to the target domain. We prove that users' privacy is secured throughout the stages involved in the proposed protocol. Experiments on both synthetic and real datasets demonstrate the efficiency of our protocol.
Interactive features of E-texts’ effects on learning: a systematic review and meta-analysis

E-texts afford interactive features that are not feasible with paper texts. Several studies have been conducted examining interactive features of E-texts, but it is uncertain what the overall effect is or what features may be most useful. The purpose of this study is to systematically review and meta-analysis of the findings comparing reading performance and/or reading times between E-texts with interactive features and control texts (paper or static E-texts). The systematic search of the literature identified 26 independent studies on reading performance. Based on the meta-analysis, interactive features benefited reading performance (g = .66, p < .001). Individual studies with positive effects involved multiple interactive features; however, potential contributions of three types of features (questions with feedback, digital glossaries, and collaborative tools) are discussed. Future directions for examining interactive features experimentally to better understand what features are most helpful for whom are described.
Intra-Block Correlation Based Reversible Data Hiding in Encrypted Images Using Parametric Binary Tree Labeling

In this paper, a high capacity reversible data hiding technique using a parametric binary tree labeling scheme is proposed. The proposed parametric binary tree labeling scheme is used to label a plaintext image’s pixels as two different categories, regular pixels and irregular pixels, through a symmetric or asymmetric process. Regular pixels are only utilized for secret payload embedding whereas irregular pixels are not utilized. The proposed technique efficiently exploits intra-block correlation, based on the prediction mean of the block by symmetry or asymmetry. Further, the proposed method utilizes blocks that are selected for their pixel correlation rather than exploiting all the blocks for secret payload embedding. In addition, the proposed scheme enhances the encryption performance by employing standard encryption techniques, unlike other block based reversible data hiding in encrypted images. Experimental results show that the proposed technique maximizes the embedding rate in comparison to state-of-the-art reversible data hiding in encrypted images, while preserving privacy of the original contents.
ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference

In this paper, we present ManyTypes4Py, a large Python dataset for machine learning (ML)-based type inference. The dataset contains a total of 5,382 Python projects with more than 869K type annotations. Duplicate source code files were removed to eliminate the negative effect of the duplication bias. To facilitate training and evaluation of ML models, the dataset was split into training, validation and test sets by files. To extract type information from abstract syntax trees (ASTs), a light-weight static analyzer pipeline is developed and accompanied with the dataset. Using this pipeline, the collected Python projects were analyzed and the results of the AST analysis were stored in JSON-formatted files. The ManyTypes4Py dataset is shared on zenodo and its tools are publicly available on GitHub.
JTSG: A joint term-sentiment generator for aspect-based sentiment analysis

This paper focuses on two related sub-tasks of aspect-based sentiment analysis, namely aspect-term extraction and aspect sentiment classification. The former aims to extract aspect-terms from given sentences and the latter aims to identify the sentiment polarity expressed on the extracted terms. Considering the practical application, researchers use more joint methods rather than pipeline methods. However, existing joint methods cannot model the interaction between aspect-terms and the sentence they belong to, or consider the relevance among the sentiments of different aspect-terms. In this paper, a novel end-to-end generative model based on encoder-decoder, namely Joint Term-Sentiment Generator (JTSG), is presented to generate all aspect term-polarity pairs. Specifically, a pre-trained model based encoder is used to encode the sentences, and specially, the decoder generates the start and end position to determine an aspect-term, rather than generate aspect-terms themselves. This new generative method contributes to avoid generating incomplete aspect-terms. Experimental results demonstrate that the proposed approach yields competitive performance on three benchmark datasets.
Feature-Level Collaboration: Joint Unsupervised Learning of Optical Flow, Stereo Depth and Camera Motion

Precise estimation of optical flow, stereo depth and camera motion are important for the real-world 3D scene understanding and visual perception. Since the three tasks are tightly coupled with the inherent 3D geometric constraints, current studies have demonstrated that the three tasks can be improved through jointly optimizing geometric loss functions of several individual networks. In this paper, we show that effective feature-level collaboration of the networks for the three respective tasks could achieve much greater performance improvement for all three tasks than only loss-level joint optimization. Specifically, we propose a single network to combine and improve the three tasks. The network extracts the features of two consecutive stereo images, and simultaneously estimates optical flow, stereo depth and camera motion. The whole network mainly contains four parts: (I) a feature-sharing encoder to extract features of input images, which can enhance features’ representation ability; (II) a pooled decoder to estimate both optical flow and stereo depth; (III) a camera pose estimation module which fuses optical flow and stereo depth information; (IV) a cost volume complement module to improve the performance of optical flow in static and occluded regions. Our method achieves state-of-the-art performance among the joint unsupervised methods, including optical flow and stereo depth estimation on KITTI 2012 and 2015 benchmarks, and camera motion estimation on KITTI VO dataset.
Performance of Regression Models as a Function of Experiment Noise

A challenge in developing machine learning regression models is that it is difficult to know whether maximal performance has been reached on the test dataset, or whether further model improvement is possible. In biology, this problem is particularly pronounced as sample labels (response variables) are typically obtained through experiments and therefore have experiment noise associated with them. Such label noise puts a fundamental limit to the metrics of performance attainable by regression models on the test dataset.We address this challenge by deriving an expected upper bound for the coefficient of determination (R2) for regression models when tested on the holdout dataset. This upper bound depends only on the noise associated with the response variable in a dataset as well as its variance. The upper bound estimate was validated via Monte Carlo simulations and then used as a tool to bootstrap performance of regression models trained on biological datasets, including protein sequence data, transcriptomic data, and genomic data.The new method for estimating upper bounds for model performance on test data should aid researchers in developing ML regression models that reach their maximum potential. Although we study biological datasets in this work, the new upper bound estimates will hold true for regression models from any research field or application area where response variables have associated noise.
Deep Learning-based Distortion Sensitivity Prediction for Full-Reference Image Quality Assessment

Previous full-reference image quality assessment methods aim to evaluate the quality of images impaired by traditional distortions such as JPEG, white noise, Gaussian blur, and so on. However, there is a lack of research measuring the quality of images generated by various image processing algorithms, including super-resolution, denoising, restoration, etc. Motivated by the previous model that predicts the distortion sensitivity maps, we use the DeepQA as a baseline model on a challenge database that includes various distortions. We have further improved the baseline model by dividing it into three parts and modifying each: 1) distortion encoding network, 2) sensitivity generation network, and 3) score regression. Through rigorous experiments, the proposed model achieves better prediction accuracy on the challenge database than other methods. Also, the proposed method shows better visualization results compared to the baseline model. We submitted our model in NTIRE 2021 Perceptual Image Quality Assessment Challenge and won 12th in the main score.
Toward Interactive Modulation for Photo-Realistic Image Restoration

Modulating image restoration level aims to generate a restored image by altering a factor that represents the restoration strength. Previous works mainly focused on optimizing the mean squared reconstruction error, which brings high reconstruction accuracy but lacks finer texture details. This paper presents a Controllable Unet Generative Adversarial Network (CUGAN) to generate high-frequency textures in the modulation tasks. CUGAN consists of two modules - base networks and condition networks. The base networks comprise a generator and a discriminator. In the generator, we realize the interactive control of restoration levels by tuning the weights of different features from different scales in the Unet architecture. Moreover, we adaptively modulate the intermediate features in the discriminator according to the severity of degradations. The condition networks accept the condition vector (encoded degradation information) as input, then generate modulation parameters for both the generator and the discriminator. During testing, users can control the output effects by tweaking the condition vector. We also provide a smooth transition between GAN and MSE effects by a simple transition method. Extensive experiments demonstrate that the proposed CUGAN achieves excellent performance on image restoration modulation tasks.
MUST-GAN: Multi-level Statistics Transfer for Self-driven Person Image Generation

Pose-guided person image generation usually involves using paired source-target images to supervise the training, which significantly increases the data preparation effort and limits the application of the models. To deal with this problem, we propose a novel multi-level statistics transfer model, which disentangles and transfers multi-level appearance features from person images and merges them with pose features to reconstruct the source person images themselves. So that the source images can be used as supervision for self-driven person image generation. Specifically, our model extracts multi-level features from the appearance encoder and learns the optimal appearance representation through attention mechanism and attributes statistics. Then we transfer them to a pose-guided generator for re-fusion of appearance and pose. Our approach allows for flexible manipulation of person appearance and pose properties to perform pose transfer and clothes style transfer tasks. Experimental results on the DeepFashion dataset demonstrate our method’s superiority compared with state-of-the-art supervised and unsupervised methods. In addition, our approach also performs well in the wild.
Beyond the Horizon: Integrating Immersive Learning Environments in the Everyday Classroom

As Immersive Learning research gains traction, questions arise about how educational Virtual and Augmented Realities can be transferred from laboratory settings and pilot projects into everyday teaching. This paper analyzes existing pedagogical frameworks to identify influencing factors and challenges relevant to teaching and learning with immersive learning environments. We distinguish Immersive Learning as individual learning processes supported by immersive technology and Immersive Teaching as the process of teaching with immersive technology. We subsume learner-specific influences (micro-level), teacher- and classroom-specific influences (meso-level) and institutional and governmental factors (macro-level) for Immersive Teaching and Learning. We conclude that, while investigating isolated variables is important for basic research, efforts integrating Virtual and Augmented Reality in everyday classrooms raise new challenges and questions for future research on the complex relationship between various factors.
Self-Aligned Video Deraining with Transmission-Depth Consistency

In this paper, we address the problem of rain streaks and rain accumulation removal in video, by developing a self-alignment network with transmission-depth consistency. Existing video based deraining methods focus only on rain streak removal, and commonly use optical flow to align the rain video frames. However, besides rain streaks, rain accummulation can considerably degrade visibility; and, optical flow estimation in a rain video is still erroneous, making the deraining performance tend to be inaccurate. Our method employs deformable convolution layers in our encoder to achieve feature-level frame alignment, and hence avoids using optical flow. For rain streaks, our method predicts the current frame from its adjacent frames, such that rain streaks that appear randomly in the temporal domain can be removed. For rain accumulation, our method employs a transmission-depth consistency loss to resolve the ambiguity between the depth and water-droplet density. Our network estimates the depth from consecutive rain-accumulation-removal outputs, and calculates the transmission map using a commonly used physics model. To ensure photometric-temporal and depth-temporal consistencies, our method estimates the camera poses, so that it can warp one frame to its adjacent frames. Experimental results show that our method is effective in removing both rain streaks and rain accumulation, outperforming those of state-of-the-art methods quantitatively and qualitatively.