Reviewerly/exHarmony huggingface dataset:
Privacy-Preserving Text Classification on BERT Embeddings with Homomorphic Encryption

Embeddings, which compress information in raw text into semantics-preserving lowdimensional vectors, have been widely adopted for their efficacy.However, recent research has shown that embeddings can potentially leak private information about sensitive attributes of the text, and in some cases, can be inverted to recover the original input text.To address these growing privacy challenges, we propose a privatization mechanism for embeddings based on homomorphic encryption, to prevent potential leakage of any piece of information in the process of text classification.In particular, our method performs text classification on the encryption of embeddings from stateof-the-art models like BERT, supported by an efficient GPU implementation of CKKS encryption scheme.We show that our method offers encrypted protection of BERT embeddings, while largely preserving their utility on downstream text classification tasks.
On the Effectiveness of Sentence Encoding for Intent Detection Meta-Learning

Recent studies on few-shot intent detection have attempted to formulate the task as a metalearning problem, where a meta-learning model is trained with a certain capability to quickly adapt to newly specified few-shot tasks with potentially unseen intent categories.Prototypical networks have been commonly used in this setting, with the hope that good prototypical representations could be learned to capture the semantic similarity between the query and a few labeled instances.This intuition naturally leaves a question of whether or not a good sentence representation scheme could suffice for the task without further domain-specific adaptation.In this paper, we conduct empirical studies on a number of general-purpose sentence embedding schemes, showing that good sentence embeddings without any fine-tuning on intent detection data could produce a nontrivially strong performance.Inspired by the results from our qualitative analysis, we propose a frustratingly easy modification, which leads to consistent improvements over all sentence encoding schemes, including those from the state-of-the-art prototypical network variants with task-specific fine-tuning. 1
GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers

There has been a growing interest in interpreting the underlying dynamics of Transformers.While self-attention patterns were initially deemed as the primary option, recent studies have shown that integrating other components can yield more accurate explanations.This paper introduces a novel token attribution analysis method that incorporates all the components in the encoder block and aggregates this throughout layers.Through extensive quantitative and qualitative experiments, we demonstrate that our method can produce faithful and meaningful global token attributions.Our experiments reveal that incorporating almost every encoder component results in increasingly more accurate analysis in both local (single layer) and global (the whole model) settings.Our global attribution analysis significantly outperforms previous methods on various tasks regarding correlation with gradient-based saliency scores.Our code is freely available at https://github.com/mohsenfayyaz/GlobEnc.⋆ Equal contribution. 1 We also have shown the unreliability of weights due to norm disparities in probing studies (Fayyaz et al., 2021).
Toward Robust Networks against Adversarial Attacks for Radio Signal Modulation Classification

Deep learning (DL) is a powerful technique for many real-time applications, but it is vulnerable to adversarial attacks. Herein, we consider DL-based modulation classification, with the objective to create DL models that are robust against attacks. Specifically, we introduce three defense techniques: i) randomized smoothing, ii) hybrid projected gradient descent adversarial training, and iii) fast adversarial training, and evaluate them under both white-box (WB) and black-box (BB) attacks. We show that the proposed fast adversarial training is more robust and computationally efficient than the other techniques, and can create models that are extremely robust to practical (BB) attacks.
A Critical Review of the Applications and AI Techniques for Anomaly Detection

In the process of analysing data, outlier detection (i.e., anomaly detection or novelty identification) is often misinterpreted to an identification of rare observations, occurrence or an item, which deviates highly from enormous data and never conforms to well- defined ideologies of a normal behaviour. The samples could stimulate more suspicion of being produced from various techniques, or appear unpredictable with the remaining portion of the specific dataset. Anomaly detection draws application in different domains such as neuroscience, statistics, machine vision, medicine, financial fraud, law enforcement and cyber security. The data that has been collected from real-life applications are rapidly increasing in dimension and size. As the aspect of dimensionality keeps increasing, data items become significantly sparse, amounting to an identification of variances becoming problematic. In addition, more conventional approaches for anomaly detection cannot function in a proper manner. In this paper, we have evaluated the applications and methods of anomaly detection.
Waste Collection and Transportation Supervision Based on Improved YOLOv3 Model

Supervising the process of garbage collection and transportation is a very crucial task with significance implication to the reusing and the recycle of the garbage.Previous works mainly focus on the detection of certain kind of garbage instead of overseeing the whole process.This paper proposed a supervision approach based on the improved YOLOv3 to decrease the incidence of unexpected mixing of the different kinds of garbage during the transportation and the collection which will cause the inferior performance of waste classification.Firstly, to reduce the parameters and arithmetic operations of YOLOv3 and improving the model's detection speed, we displaced the standard convolution in YOLOv3 by depthwise separable convolution.Secondly, to solve the problem that YOLOv3 has poor location accuracy and performs poorly in muti-targets, triplet attention is introduced into the backbone, which increases almost no parameters to automatically learn cross-dimensional interactions, enhance the effective feature channel weights, and strengthen the feature extraction capability.Finally, we built a dataset of waste classification supervision using the images provided by a city environmental protection bureau on which we did massive experiments.The experimental result shows that compared with other detection algorithms, the improved YOLO v3 algorithm has better performance.The mAP is 98.5%, which is 0.7% and 1.1% higher than the YOLOv5l and the EfficientDet-B0, respectively, and the average detection speed of the model is 14.6ms/it, which meets the requirements of regulatory real-time and environment complexity of the supervision system.
Meningkatkan Kemampuan Mengenal Angka 1-10 pada Anak Usia 4-5 Tahun Melalui Kegiatan Bermain Variatif dengan Media Loose Part

This study aims to improve the ability of children to recognize numbers 1-10 in group A TK Sinar Kasih Sangup, Kecamatan Tamansari, Kabupaten Boyolali. This research uses Classroom Action Research (CAR). The research subjects in group A of Sinar Kasih Sangup Kindergarten were 10 children consisting of 6 girls and 4 boys. Data collection techniques in this study are observation and documentation. The data analysis technique used descriptive analysis. The results of this study indicate an increase in children's ability to recognize numbers 1-10 through varied play activities with loose part media, namely in the pre-cycle the ability of children to recognize numbers 1-10 is only 20%, at the first cycle meeting children's ability to recognize numbers 1-10 increases to 40%, in cycle 2 the child's ability to recognize numbers 1-10 increased to 60%, and in cycle 3 the child's ability to recognize numbers 1-10 increased to 80%. The indicator used in this study is that children are able to recognize numbers 1-10. The conclusion of this study is that children are able to recognize numbers 1-10 with varied play activities with loose part media.
Issues in Sub-Utterance Level Language Identification in a Code Switched Bilingual Scenario

Sub-utterance level language identification (SLID) is an automatic process of recognizing the spoken language in a code switched (CS) utterance at the sub-utterance level. The nature of CS utterances suggest the primary language has a significant duration of occurrence over the secondary. In a CS utterance, a single speaker speaks both the languages. Hence the phoneme-level acoustic characteristic (sub-segmental and segmental evidence) of the secondary language is mostly biased towards the primary. This hypothesizes that the acoustic-based language identification system using CS training data may end with a biased performance towards the primary language. This study proves the hypothesis by observing the performance in terms of the confusion matrix of the earlier proposed approaches. At the same time, language discrimination also can be done at the suprasegmental-level, by capturing language-specific phonemic temporal evidence. Hence, to resolve the biasing issue, this study proposes a wav2vec2-based approach, which captures suprasegmental phonemic temporal patterns in the pre-training stage and merges it to capture language-specific suprasegmental evidence in the finetuning stage. The experimental results show the proposed approach is able to resolve the issue to some extent. As the fine-tuning stage uses a discriminative approach, the weighted loss and secondary language augmentation methods can be explored in the future for further performance improvement.
A self-interpretable module for deep image classification on small data

Abstract Deep neural networks are the driving force of the recent explosion of machine learning applications in everyday life. However, they usually require a lot of training data to work well, and they act as black-boxes, making predictions without any explanation about them. This paper presents Memory Wrap, a module (i.e, a set of layers) that can be added to deep learning models to improve their performance and interpretability in settings where few data are available. Memory Wrap adopts a sparse content-attention mechanism between the input and some memories of past training samples. We show that adding Memory Wrap to standard deep neural networks improves their performance when they learn from a limited set of data, and allows them to reach comparable performance when they learn from the full dataset. We discuss how the analysis of its structure and content-attention weights helps to get insights about its decision process and makes their predictions more interpretable, compared to the same networks without Memory Wrap. We test our approach on image classification tasks using several networks on three different datasets, namely CIFAR10, SVHN, and CINIC10.
Blockchain Socket Factories with RMI-Enabled Framework for Fine-Grained Healthcare Applications

The usage of digital and intelligent healthcare applications on mobile devices has grown progressively. These applications are generally distributed and access remote healthcare services on the user’s applications from different hospital sources. These applications are designed based on client–server architecture and different paradigms such as socket, remote procedure call, and remote method invocation (RMI). However, these existing paradigms do not offer a security mechanism for healthcare applications in distributed mobile-fog-cloud networks. This paper devises a blockchain-socket-RMI-based framework for fine-grained healthcare applications in the mobile-fog-cloud network. This study introduces a new open healthcare framework for applied research purposes and has blockchain-socket-RMI abstraction level classes for healthcare applications. The goal is to meet the security and deadline requirements of fine-grained healthcare tasks and minimize execution and data validation costs during processing applications in the system. This study introduces a partial proof of validation (PPoV) scheme that converts the workload into the hash and validates it among mobile, fog, and cloud nodes during offloading, execution, and storing data in the secure form. Simulation discussions illustrate that the proposed blockchain-socket-RMI minimizes the processing and blockchain costs and meets the security and deadline requirements of fine-grained healthcare tasks of applications as compared to existing frameworks in work.
Towards lifelong learning of Recurrent Neural Networks for control design

This paper proposes a method for lifelong learning of Recurrent Neural Networks, such as NNARX, ESN, LSTM, and GRU, to be used as plant models in control system synthesis.The problem is significant because in many practical applications it is required to adapt the model when new information is available and/or the system undergoes changes, without the need to store an increasing amount of data as time proceeds.Indeed, in this context, many problems arise, such as the well known Catastrophic Forgetting and Capacity Saturation ones.We propose an adaptation algorithm inspired by Moving Horizon Estimators, deriving conditions for its convergence.The described method is applied to a simulated chemical plant, already adopted as a challenging benchmark in the existing literature.The main results achieved are discussed.
Multiclass Classification Performance Curve

Quality of predictive models is a critical factor. Many evaluation measures have been proposed for binary and multi&#x2013;class datasets. However, less attention has been paid to graphical representation of the classification performance, where the ROC curve is extensively used for binary datasets but there is no standard method accepted by the scientific community for multi&#x2013;class datasets. In this work, a multi&#x2013;class classification performance (MCP) curve based on the Hellinger distance between true and prediction probabilities of the classifier is introduced. The MCP curve shows the classification performance, contributes to highlight the low or high confidence on correct predictions, and quantifies the quality by means of the area under the curve.
Codebook-softened product quantization for high accuracy approximate nearest neighbor search

Product quantization (PQ) is a fundamental technique for approximate nearest neighbor (ANN) search in many applications such as information retrieval, computer vision and pattern recognition. In the existing PQ-based methods for approximate nearest neighbor search, the reachable best search accuracy is achieved by using fixed codebooks. The search performance is limited by the quality of the hard codebooks. Unlike the existing methods, in this paper, we present a novel codebook-softened product quantization (CSPQ) method to achieve more quantization levels by softening codebooks. We firstly analyze how well the database vectors match the trained codebooks by examining quantization error for each database vector, and select the bad-matching database vectors. Then, we give the trained codebooks b-bit freedom to soften codebooks. Finally, by minimizing quantization errors, the bad-matching vectors are encoded by softened codebooks and the labels of best-matching codebooks are recorded. Experimental results on SIFT1M, GIST1M and SIFT10M show that, despite its simplicity, our proposed method achieves higher accuracy compared with PQ and it can be combined with other non-exhaustive frameworks to achieve fast search.
Coherent Ising machine with quantum feedback: The total and conditional master equation methods

We give a detailed theoretical derivation of the master equation for the coherent Ising machine. This is a quantum computational network with feedback, that approximately solves NP-hard combinatoric problems, including the traveling salesman problem and various extensions and analogs. There are two possible types of master equation, either conditional on the feedback current or unconditional. We show that both types can be accurately simulated in a scalable way using stochastic equations in the positive-P phase-space representation. This depends on the nonlinearity present, and we use parameter values that are typical of current experiments. While the two approaches are in excellent agreement, they are not equivalent with regard to efficiency. We find that unconditional simulation has much greater efficiency, and is more scalable to large sizes. This is a case where too much knowledge is a dangerous thing. Conditioning the simulations on the feedback current is not essential to determining the success probability, but it greatly increases the computational complexity. To illustrate the speed improvements obtained with the unconditional approach, we carry out full quantum simulations of the master equation with up to 1000 nodes.
Hardware-Aware Evolutionary Filter Pruning

Compression techniques for Convolutional Neural Networks (CNNs) are key to performance. One common technique is filter pruning, which can effectively reduce the memory footprint, number of arithmetic operations, and consequently inference time. Recently, several approaches have been presented for automatic CNN compression using filter pruning, where the number of pruned filters is optimized by nature-inspired metaheuristics (e.g., artificial bee colony algorithms). However, these approaches focus on finding an optimal pruned network structure without considering the targeted device for CNN deployment. In this work, we show that the typical objective of reducing the number of operations does not necessarily lead to a maximum reduction in inference time, which is usually the main goal for compressing CNNs besides reducing the memory footprint. We then propose a hardware-aware multi-objective Design Space Exploration (DSE) technique for filter pruning that involves the targeted device (i.e., Graphics Processing Units (GPUs)). For each layer, the number of filters to be pruned is optimized with the objectives of minimizing the inference time and the error rate of the CNN. Experimental results show that our approach can further speed up inference time by $$1.24\times $$ and $$1.09\times $$ for VGG-16 on the CIFAR-10 dataset and ResNet-101 on the ILSVRC-2012 dataset, respectively, compared to the state-of-the-art ABCPruner.
On the Usage of Neural POS Taggers for Shakespearean Literature in Social Systems

Part-of-speech (POS) taggers are the primary requisite of any natural language processing (NLP) mechanism. Conventional POS tagger and libraries are expert-made or static and concentrate on the literature domain. These POS taggers limit the performance of subsequent mechanisms like polarity detection, sentiment analysis, opinion mining, and so on. The unsuitability of a tagger for a new genre of literature makes famous libraries, such as Natural Language Toolkit (NLTK) and University Centre for Computer Corpus Research on Language (UCREL) Constituent Likelihood Automatic Word-tagging System Seven (CLAWS7) create the need for a neural POS tagger to serve Shakespearean literature. This article reports a preliminary study on the suitability of the neural taggers over static or manual taggers, supported by the accuracy of 97% achieved on <i>Hamlet</i>. Furthermore, these neural networks are scalable over the literature domains irrespective of the stylistic variations, opening up this area to computer scientists to aid literary enthusiasts in contributing to domain of the social systems.
Debiasing Neighbor Aggregation for Graph Neural Network in Recommender Systems

Graph neural networks (GNNs) have achieved remarkable success in recommender systems by representing users and items based on their historical interactions. However, little attention was paid to GNN's vulnerability to exposure bias: users are exposed to a limited number of items so that a system only learns a biased view of user preference to result in suboptimal recommendation quality. Although inverse propensity weighting is known to recognize and alleviate exposure bias, it usually works on the final objective with the model outputs, whereas GNN can also be biased during neighbor aggregation. In this paper, we propose a simple but effective approach, neighbor aggregation via inverse propensity (NAVIP) for GNNs. Specifically, given a user-item bipartite graph, we first derive propensity score of each user-item interaction in the graph. Then, inverse of the propensity score with Laplacian normalization is applied to debias neighbor aggregation from exposure bias. We validate the effectiveness of our approach through our extensive experiments on two public and Amazon Alexa datasets where the performance enhances up to 14.2%.
Integrating Transformer and GCN for COVID-19 Forecasting

The spread of corona virus disease 2019 (COVID-19) has coincided with the rise of Transformer and graph neural networks, leading several studies to propose using them to better predict the evolution of a pandemic. The inconveniences of infectious diseases make it important to predict their spread. However, the single deep learning (DL) model has the problems of unstable prediction effect and poor convergence. When calculating the relationship between different positions within a sequence, Transformer does not consider the local context in which each position is located, which can make the prediction vulnerable to outliers, so the integration of the graph convolutional network (GCN) to capture local information is considered. In this paper, we use Transformer to encode the time sequence information of COVID-19 and GCN to decode the time sequence information with graph structure, so that Transformer and GCN are perfectly combined and spatial information is used to further study the integration of these two methods. In addition, we improve the traditional positional encoding structure and propose a dynamic positional encoding technique to extract dynamic temporal information effectively, which is proved to be the key to capture spatial and temporal patterns in data. To make our predictions more useful, we only focused on three states in the United States, covering one of the most affected states, one of the least affected states, and one intermediate state. We used mean absolute percentage error and mean square error as evaluation indexes. Experimental results show that the proposed time series model has better predictive performance than the current DL models. Moreover, the convergence of our model is also better than the current DL models, providing a more accurate reference for the prevention of epidemics.