Reviewerly/exHarmony huggingface dataset:
Feasibility of Privacy-Preserving Federated Deep Learning on Medical Images

Deep learning (DL) has immense potential to revolutionize healthcare. Several Federated DL solutions have been proposed to access massive repositories of private data, without transferring subject data from the host devices. There remain concerns about potential privacy violation via “reconstructing” individually-identifiable subject data by exploiting model weights from one host. We propose a methodology for federated DL that addresses this risk through cloud-server architecture design. The study is a fully federated replication of a previous DM classifier derived on 300 head-and-neck cancer patients. Only CTs (1 slice from mid-tumor) from a The Cancer Imaging Archive public collection was used. Two hospital cohorts were used for training, and 2 other cohorts were used as validation. A “privacy-by-architecture” federated DL prototype was developed by extending an open-source distributed learning system VANTAGE6. An authenticated researcher submitted a learning task to a master server, that distributed a DL network to be trained to data hosts. Host devices were Ubuntu machines with 1 GPU each and data stored in its local XNAT repository. Training and validation were randomly split across both data hosts. The DL model consisted of 3 fully-connected convolutional blocks and 1 sigmoid output layer. In each iteration of training, synchronous averaging and update of model weights were managed through a separate aggregation server connected to each data host via a key and encrypted web interfaces. The aggregation server prevented direct connections between hosts, researcher and master server while training, so that none could peek at interim model weights generated by others. When finished, final globally-averaged weights were stored on the master server for retrieval by the researcher, but aggregator contents were erased. We demonstrated functional equivalence of a federated DL classifier to its centralized version, without transferring any subject-level data from data host machines. Further, by preventing access to interim model weights, we added an additional layer of privacy protection which we call "privacy-by-architecture". The centralized learning and federated learning models delivered statistically equivalent performance in terms of accuracy, loss and area under receiver-operator curve. 5-fold cross validation mean AUC/accuracy of centralized and federated DL models are 0.88 vs 0.85 and 0.88 vs 0.86 respectively. We demonstrated a privacy-preserving federated DL methodology that enhances privacy protection by preventing visibility of model weights during training; globally-averaged model weights reduce the risk of privacy violation by ” reconstruction” attacks on the data hosts. Our architecture may be further enhanced by adding differential privacy and encryption in future.
Reversible Watermarking in Medical Images Using Sub-Sample and Multiple Histogram Modification

This paper proposed a reversible medical image watermarking scheme using multiple histogram modification (MHM) and redundant discrete wavelet transform (RDWT). The MHM was introduced to the proposed scheme to enhance the embedding capacity. By embedding the watermark in the RDWT coefficients, the proposed scheme exploited the visual masking property of RDWT to guarantee the visual quality. Also, the proposed scheme has better performance on embedding capacity because the RDWT has several sub-band coefficients for embedding. The experimental results on medical images suggests that the proposed scheme could meet the demand of perceptional quality with better embedding capacity than former schemes.
A Deep Learning Approach for the Romanized Tunisian Dialect Identification

Language identification is an important task in natural language processing that consists in determining the language of a given text. It has increasingly picked the interest of researchers for the past few years, especially for code-switching informal textual content. In this paper, we focus on the identification of the Romanized user-generated Tunisian dialect on the social web. We segment and annotate a corpus extracted from social media and propose a deep learning approach for the identification task. We use a Bidirectional Long Short-Term Memory neural network with Conditional Random Fields decoding (BLSTM-CRF). For word embeddings, we combine word-character BLSTM vector representation and Fast Text embeddings that takes into consideration character n-gram features. The overall accuracy obtained is 98.65%.
Automatic Recognition of Anomalous Patterns in Discharges by Applying Deep Learning

Anomaly detection addresses the problem of finding unexpected values in data sets. Often, these anomalies, also known as outliers, discordant values, or exceptions, describe patterns in the behavior of the data. Anomaly detection is important because it frequently involves significant and critical information in many application domains. In the case of nuclear fusion, there is a wide variety of anomalies that could be related to plasma behaviors, such as disruptions or low-high (L-H) transitions. In this context, there are known and unknown anomalies, where unknown anomalies represent the largest proportion of the total that can be found in nuclear fusion. This paper presents a study of the application of deep learning and architecture called Autoencoder to detect anomalies predicting (encode-decode) in a discharge.
Using video conferencing services in network educational communities: Theory and experience of implementation in teaching informatics

The article raises the problem of using video conferencing services in network educational communities in the context of virtualization of the educational space when organizing the interaction of students in the process of online learning in informatics. Changes in the education system associated with the development of social networks and network communities are discussed, and new opportunities provided by them to the teacher for including students in various types of educational and cognitive activities are analyzed. The concepts of “network community” and “social networks” are revealed. The role of network communities and social networks as collective subjects of social information and educational activities on the Internet is demonstrated. From the standpoint of the implementation of the principle of interactivity of interaction of subjects of educational activity in network communities, the analysis of educational interaction of schoolchildren in the format of video conference based on the Internet is carried out. The characteristics of the capabilities of popular services for network interaction of participants in the educational process when organizing training in the video conference format are described, examples of open software solutions are given. The experience of using video conferencing services in teaching informatics and robotics in the context of the implementation of the educational process in the networked educational communities of the Internet is revealed. The author's approach to the use of tools of popular services for organizing video conferencing in the implementation of interactive training in promising areas of informatics and robotics in the process of joint activities on the Internet is proposed. The results obtained contribute to the theory of informatization of education and can be used in practice by teachers who implement the educational process with schoolchildren using interactive interaction services and Internet resources.
In Defense of Feature Mimicking for Knowledge Distillation.

Knowledge distillation (KD) is a popular method to train efficient networks (student) with the help of high-capacity networks (teacher). Traditional methods use the teacher's soft logit as extra supervision to train the student network. In this paper, we argue that it is more advantageous to make the student mimic the teacher's features in the penultimate layer. Not only the student can directly learn more effective information from the teacher feature, feature mimicking can also be applied for teachers trained without a softmax layer. Experiments show that it can achieve higher accuracy than traditional KD. To further facilitate feature mimicking, we decompose a feature vector into the magnitude and the direction. We argue that the teacher should give more freedom to the student feature's magnitude, and let the student pay more attention on mimicking the feature direction. To meet this requirement, we propose a loss term based on locality-sensitive hashing (LSH). With the help of this new loss, our method indeed mimics feature directions more accurately, relaxes constraints on feature magnitudes, and achieves state-of-the-art distillation accuracy.
AirNN: A Featherweight Framework for Dynamic Input-Dependent Approximation of CNNs

In this work, we propose AirNN, a novel framework which enables dynamic approximation of an already-trained convolutional neural network (CNN) in hardware during inference. AirNN enables input-dependent approximation of the CNN to achieve energy saving without much degradation in its classification accuracy at runtime. For each input, AirNN uses only a fraction of the CNN’s weights based on that input (with the rest remaining 0) to conduct the inference. Consequently, energy saving is possible due to fewer number of fetches from off-chip memory as well as fewer multiplications for majority of the inputs. To achieve per-input approximation, we propose a clustering algorithm that groups similar weights in the CNN based on their importance, and design an iterative framework that decides dynamically how many clusters of weights should be fetched from off-chip memory for each individual input. We also propose new hardware structures to implement our framework on top of a recently proposed FPGA-based CNN accelerator. In our experiments with popular CNNs, we, on average, show 49% energy saving with less than 3% degradation in classification accuracy due to doing inference with only a fraction of the weights for the majority of the inputs. We also propose a greedy interleaving scheme, implemented in hardware, in order to improve the performance of the iterative procedure and compensate for its latency overhead.
Compressing Genomic Sequences by Using Deep Learning

Huge amount of genomic sequences have been generated with the development of high-throughput sequencing technologies, which brings challenges to data storage, processing, and transmission. Standard compression tools designed for English text are not able to compress genomic sequences well, so an effective dedicated method is needed urgently. In this paper, we propose a genomic sequence compression algorithm based on a deep learning model and an arithmetic encoder. The deep learning model is structured as a convolutional layer followed by an attention-based bi-directional long short-term memory network, which predicts the probabilities of the next base in a sequence. The arithmetic encoder employs the probabilities to compress the sequence. We evaluate the proposed algorithm with various compression approaches, including a state-of-the-art genomic sequence compression algorithm DeepDNA, on several real-world data sets. The results show that the proposed algorithm can converge stably and achieves the best compression performance which is even up to 3.7 times better than DeepDNA. Furthermore, we conduct ablation experiments to verify the effectiveness and necessity of each part in the model and implement the visualization of attention weight matrix to present different importance of various hidden states for final prediction. The source code for the model is available in Github (https://github.com/viviancui59/Compressing-Genomic-Sequences).
THAI STUDENT ONLINE RESPONSES TO NATIVE SPEAKER TOWARD INDONESIAN LANGUAGE LEARNING FOR FOREIGN SPEAKERS COURSE (RESPON MAHASISWA THAILAND DALAM PEMBELAJARAN BAHASA INDONESIA BAGI PENUTUR ASING (BIPA) DENGAN PENUTUR ASLI SECARA ONLINE)

Thai student responses to Indonesian online learning with native speakers. The problem in this research is how Thai learners respond to learning Indonesian (BIPA) online with native speakers. And how is the online learning process provided by teachers and guest lecturers from STKIP PGRI West Sumatra, UPI Language Center, BPK Penabur School with the theme of learning is about introductions, family, and about the song Indonesia Raya. The purpose of conducting this research is to obtain an overview of how the response of Thai learners who study at NU to learning Indonesian online taught by natives. The method in this study uses two forms of research methods, namely the first library research method and the second with descriptive analysis with the technique used is to observe and give questionnaires to respondents and teachers about how to respond and their satisfaction in learning Indonesian natively. . The results show that this learning process improves Indonesian learning outcomes and increases the number of vocabulary it gets. As well as the response of Thai students who interact with natives shows the character of speaking well, by repeating the same theme twice at different times to practice speaking with Indonesians
Scale Invariant Mask R-CNN for Pedestrian Detection

Pedestrian detection is a challenging and active research area in computer vision. Recognizing pedestrians helps in various utility applications such as event detection in overcrowded areas, gender, and gait classification, etc. In this domain, the most recent research is based on instance segmentation using Mask R-CNN. Most of the pedestrian detection method uses a feature of different body portions for identifying a person. This feature-based approach is not efficient enough to differentiate pedestrians in real-time, where the background changing. In this paper, a combined approach of scale-invariant feature map generation for detecting a small pedestrian and Mask R-CNN has been proposed for multiple pedestrian detection to overcome this drawback. The new database was created by recording the behavior of the student at the prominent places of the engineering institute. This database is comparatively new for pedestrian detection in the academic environment. The proposed Scale-invariant Mask R-CNN has been tested on the newly created database and has been compared with the Caltech [1], INRIA [2], MS COCO [3], ETH [4], and KITTI [5] database. The experimental result shows significant performance improvement in pedestrian detection as compared to the existing approaches of pedestrian detection and instance segmentation. Finally, we conclude and investigate the directions for future research.
Unsupervised Learning Method for Encoder-Decoder-Based Image Restoration

The restoration of a corrupted image is a challenge to computer vision and image processing. In hazy, underwater and medical images, the lack of paired images lead the state of the art to synthesize datasets. The Generative Adversarial Networks (GANs) are widely used in these cases. However, computational cost and training instability are current concerns. We present an unsupervised learning algorithm that does not requires paired dataset to train encoder-decoder-like neural network for image restoration. An encoder-decoder learn to represent its input data in a latent representation and reconstruct then in the output. During the training stage, our algorithm applies the encoder-decoder output image to a degradation block that reinforces its degradation. The degraded and input images are matched using a loss function. After the training process, we obtain a restored image from the decoder. We used ill-exposed images to evaluate and validate our algorithm.
A Distance-Weighted Selection of Unlabelled Instances for Self-training and Co-training Semi-supervised Methods

The use of Semi-supervised Learning (SSL) methods have emerged as an efficient solution to smooth out the problem of availability of labelled instances. Several methods have been proposed in the literature and Self-training and Co-training are two well-known methods. The main aim is to use only a few labelled instances to define a model and to apply this model in a labelling process, in which unlabelled instances are labelled and included in the labelled set. However, the labelling process is always directly dependent on the selection of the unlabelled instances. Moreover, the selection criterion used to select and label new instances has an important effect in the performance of a semi-supervised method. In this paper, we propose a distance-weighted selection of unlabelled instances for Self-training and Co-training semi-supervised methods. In addition, we compare the standard Self-training and Co-training methods against the proposed versions of these two methods over 20 classification datasets.
Predicting Future Products Rate using Machine Learning Algorithms

Opinion mining in social networks data is considered as one of most important research areas because a large number of users interact with different topics on it. This paper discusses the problem of predicting future products rate according to users’ comments. Researchers interacted with this problem by using machine learning algorithms (e.g. Logistic Regression, Random Forest Regression, Support Vector Regression, Simple Linear Regression, Multiple Linear Regression, Polynomial Regression and Decision Tree). However, the accuracy of these techniques still needs to be improved. In this study, we introduce an approach for predicting future products rate using LR, RFR, and SVR. Our data set consists of tweets and its rate from 1:5. The main goal of our approach is improving the prediction accuracy about existing techniques. SVR can predict future product rate with a Mean Squared Error (MSE) of 0.4122, Linear Regression model predict with a Mean Squared Error of 0.4986 and Random Forest Regression can predict with a Mean Squared Error of 0.4770. This is better than the existing approaches accuracy.
Reproducible Color Gamut of Hematoxylin and Eosin Stained Images in Standard Color Spaces

A whole-slide imaging (WSI) system is a digital color imaging system used in digital pathology with the potential to substitute the conventional light microscope. A WSI system digitalizes a glass slide by converting the optical image to digital data with a scanner and then converting the digital data back to the optical image with a display. During the digital-to-optical or optical-to-digital conversion, a color space is required to define the mapping between the digital domain and the optical domain so that the numerical data of each color pixel can be interpreted meaningfully. Unfortunately, many current WSI products do not specify the designated color space clearly, which leaves the user using the universally default color space, sRGB. sRGB is a legacy color space that has a limited color gamut, which is known to be unable to reproduce all color shades present in histology slides. In this work, experiments were conducted to quantitatively investigate the limitation of the sRGB color space used in WSI systems. Eight hematoxylin and eosin (H and E)-stained tissue samples, including human bladder, brain, breast, colon, kidney, liver, lung, and uterus, were measured with a multispectral imaging system to obtain the true colors at the pixel level. The measured color truth of each pixel was converted into the standard CIELAB color space to test whether it was within the color gamut of the sRGB color space. Experiment results show that all the eight images have a portion of pixels outside the sRGB color gamut. In the worst-case scenario, the bladder sample, about 35% of the image exceeded the sRGB color gamut. The results suggest that the sRGB color space is inadequate for WSI scanners to encode H and E-stained whole-slide images, and an sRGB display may have insufficient color gamut for displaying H and E-stained histology images.
Multi Layered Multi Task Marker Based Interaction in Information Rich Virtual Environments

Simple and cheap interaction has a key role in the operation and exploration of any Virtual Environment (VE).In this paper, we propose an interaction technique that provides two different ways of interaction (information and control) on complex objects in a simple and computationally cheap way.The interaction is based on the use of multiple embedded markers in a specialized manner.The proposed marker like an interaction peripheral works just like a touch paid which can perform any type of interaction in a 3D VE.The proposed marker is not only used for interaction with Augmented Reality (AR), but also with Mixed Reality.A biological virtual learning application is developed which is used for evaluation and experimentation.We conducted our experiments in two phases.First, we compared a simple VE with the proposed layered VE.Second, a comparative study is conducted between the proposed marker, a simple layered marker, and multiple single markers.We found the proposed marker with improved learning, easiness in interaction, and comparatively less task execution time.The results gave improved learning for layered VE as compared to simple VE.
A Preprocessing by Using Multiple Steganography for Intentional Image Downsampling on CNN-Based Steganalysis

There exists a need of ''image steganalysis'' which reveals whether steganographic signals are embedded in an image to improve information security.Among various steganalysis, Convolutional Neural Networks (CNN) based steganalysis is promising since it can automatically learn the features of diverse steganographic algorithms.However, the detection performance of CNN is degraded when an image is intentionally resized by the nearest-neighbor interpolation before steganography.This is because spatial frequency in a resized image gets high, which disturbs the training.In order to overcome this shortcoming, in this article, we propose a preprocessing by using multiple steganography for intentional image downsampling on CNN-based steganalysis.In the proposed preprocessing, steganographic signals are additionally embedded into both resized original images and resized steganographic ones with the same embedding key since difference of spatial frequencies between them gets obvious, which helps CNN learn features.The reason why the difference gets obvious is that steganographic signals tend to be continuously embedded into same pixels in resized images when they are additionally embedded.Thus, by training resized images after the proposed preprocessing, the detection performance can be improved.Since the proposed preprocessing is very simple, it does not greatly increase the training time of CNN.Our evaluation shows accuracy in a model with the proposed preprocessing is up to 34.8% higher than that in the conventional model when the same steganography is additionally embedded.Besides, we also show that the proposed preprocessing yields up to 23.1% higher accuracy compared with the conventional one even when another steganography is additionally embedded.
From WiscKey to Bourbon: A Learned Index for Log-Structured Merge Trees

We introduce BOURBON, a log-structured merge (LSM) tree that utilizes machine learning to provide fast lookups. We base the design and implementation of BOURBON on empirically-grounded principles that we derive through careful analysis of LSM design. BOURBON employs greedy piecewise linear regression to learn key distributions, enabling fast lookup with minimal computation, and applies a cost-benefit strategy to decide when learning will be worthwhile. Through a series of experiments on both synthetic and real-world datasets, we show that BOURBON improves lookup performance by 1.23x-1.78x as compared to state-of-the-art production LSMs.