Reviewerly/exHarmony huggingface dataset:
Decision trees as partitioning machines to characterize their generalization properties

Decision trees are popular machine learning models that are simple to build and easy to interpret. Even though algorithms to learn decision trees date back to almost 50 years, key properties affecting their generalization error are still weakly bounded. Hence, we revisit binary decision trees on real-valued features from the perspective of partitions of the data. We introduce the notion of partitioning function, and we relate it to the growth function and to the VC dimension. Using this new concept, we are able to find the exact VC dimension of decision stumps, which is given by the largest integer $d$ such that $2\ell \ge \binom{d}{\left\lfloor\frac{d}{2}\right\rfloor}$, where $\ell$ is the number of real-valued features. We provide a recursive expression to bound the partitioning functions, resulting in a upper bound on the growth function of any decision tree structure. This allows us to show that the VC dimension of a binary tree structure with $N$ internal nodes is of order $N \log(N\ell)$. Finally, we elaborate a pruning algorithm based on these results that performs better than the CART algorithm on a number of datasets, with the advantage that no cross-validation is required.
An Overview of Blockchain and its Applications in the Modern Digital Age

Blockchain is one of the emerging technologies that has been gaining prominence in the world’s technological scenario. It is a chain of blocks that is a part of a collective registration system. This means that the information is not saved in a specific place. Instead of information being saved on a single computer, all blockchain information is distributed among the various computers connected to it. Similar to the Internet of Things, market has developed remarkably in recent years, and with it, the number of invasions in both homes, businesses, transport systems, and smart cities. Blockchain could avoid a vulnerable device from transmitting false information throughout the system, or, in the case of a network overload, the registration of blockchain technology has the ability to identify the point where something went wrong. Within this context, blockchain can be used by companies to provide reliable, decentralized, encrypted, and uneditable databases. Thus, this chapter intends to provide a review and an overview of blockchain and their applications, addressing its evolution and fundamental concepts, showing its relationship, debating its success, exposing a concise bibliographic scope, and particularizing and characterizing the potential of the technology.
Development of a Compressed Deep Neural Network for Detecting Defected Electrical Substation Insulators using a Drone

This paper proposes a method to detect defected insulators of electrical substations in real-time on an embedded graphics processing unit (GPU) by using a drone camera. The proposed algorithm is based on a compressed deep neural network to detect defected insulators with a corona effect arising from an overcurrent. A compressed deep neural network model is developed to reduce the computation time while maintaining high detection accuracy for the embedded GPU board. This paper applies the MobileNetv2 structure and compression technique, a residual block, on YOLOv3 to reduce the overall memory size and calculation time of the detection architecture. The proposed network is trained with an image data set of a mockup insulator with and without the corona effect. The performance evaluation shows that the algorithm is able to detect the object with an accuracy of 85% at an average speed of 40 FPS on a NVIDIA Xavier board. In addition, the results show that the proposed detection network uses half the memory and less computation time than YOLOv3-Tiny.
On convergence of nearest neighbor classifiers over feature transformations

The k-Nearest Neighbors (kNN) classifier is a fundamental non-parametric machine learning algorithm. However, it is well known that it suffers from the curse of dimensionality, which is why in practice one often applies a kNN classifier on top of a (pre-trained) feature transformation. From a theoretical perspective, most, if not all theoretical results aimed at understanding the kNN classifier are derived for the raw feature space. This leads to an emerging gap between our theoretical understanding of kNN and its practical applications. In this paper, we take a first step towards bridging this gap. We provide a novel analysis on the convergence rates of a kNN classifier over transformed features. This analysis requires in-depth understanding of the properties that connect both the transformed space and the raw feature space. More precisely, we build our convergence bound upon two key properties of the transformed space: (1) safety -- how well can one recover the raw posterior from the transformed space, and (2) smoothness -- how complex this recovery function is. Based on our result, we are able to explain why some (pre-trained) feature transformations are better suited for a kNN classifier than other. We empirically validate that both properties have an impact on the kNN convergence on 30 feature transformations with 6 benchmark datasets spanning from the vision to the text domain.
Automatic Text Localization in Scene Images: A Transfer Learning Based Approach

In this paper, a novel procedure of text localization has been discussed using deep neural network which is based on transfer learning. The set of images are fed to deep neural network framework using pre-trained model of EAST text detection and blobs are detected which are sent to the two convolution layers to measure the probability of text and non-text regions and coordinates of bounding boxes extraction respectively. By using non-maximum suppression the tight-fitting boundary box is obtained. Two publicly available datasets named, ICDAR 2003 and SVT have been employed to test the efficiency of our method and it has been observed that our method gives encouraging accuracy comparing with other existing work.
Improving Text Generation Evaluation with Batch Centering and Tempered Word Mover Distance

Recent advances in automatic evaluation metrics for text have shown that deep contextualized word representations, such as those generated by BERT encoders, are helpful for designing metrics that correlate well with human judgements. At the same time, it has been argued that contextualized word representations exhibit sub-optimal statistical properties for encoding the true similarity between words or sentences. In this paper, we present two techniques for improving encoding representations for similarity metrics: a batch-mean centering strategy that improves statistical properties; and a computationally efficient tempered Word Mover Distance, for better fusion of the information in the contextualized word representations. We conduct numerical experiments that demonstrate the robustness of our techniques, reporting results over various BERT-backbone learned metrics and achieving state of the art correlation with human ratings on several benchmarks.
Online Multi-objective Subspace Clustering for Streaming Data

This paper develops an online subspace clustering technique which is capable of handling continuous arrival of data in a streaming manner. Subspace clustering is a technique where the subset of features that are used to represent a cluster are different for different clusters. Most of the streaming data clustering methods primarily optimize only a single objective function which limits the model in capturing only a particular shape or property. However, the simultaneous optimization of multiple objectives helps in overcoming the above mentioned limitations and enables to generate good quality clusters. Inspired by this, the developed streaming subspace clustering method optimizes multiple objectives capturing cluster compactness and feature relevancy. In this paper, we consider an evolutionary-based technique and optimize multiple objective functions simultaneously to determine the optimal subspace clusters. The generated clusters in the proposed method are allowed to contain overlapping of objects. To establish the superiority of using multiple objectives, the proposed method is evaluated on three real-life and three synthetic data sets. The results obtained by the proposed method are compared with several state-of-the-art methods and the comparative study shows the superiority of using multiple objectives in the proposed method.
CMBIoV: Consensus Mechanism for Blockchain on Internet of Vehicles

Both blockchain and Internet of Vehicles (IoV) are emerging technologies that attract interest from academia and industry. Deploying blockchain peers on vehicles enables corporation, coordination, trust, security, and privacy in the IoV. However, the peers are deployed on resource-constrained on-board computing devices, and have high mobility, which causes significant challenges on the blockchain. The blockchain’s consensus mechanism must be efficient and energy-saving. In this paper, a proof-of-stake consensus mechanism is presented that uses tickets and sorters to address this issue. The proposed consensus mechanism prevents users from controlling the blockchain by hoarding a large number of coins, and reduces the cost of message communication during the election of the leader and verifiers. The performance of this mechanism could be further improved if the IoV was built over 5G communication channels.
CRUR: coupled-recurrent unit for unification, conceptualization and context capture for language representation - a generalization of bi directional LSTM

In this work we have analyzed a novel concept of sequential binding based learning capable network based on the coupling of recurrent units with Bayesian Prior definition. The coupling structure encodes to generate efficient tensor representations that can be decoded to generate efficient sentences and can describe certain events. These descriptions are derived from structural representations of visual features of images and media. An elaborated study of the different types of coupling recurrent structures are studied and some insights of their performance are provided. Supervised learning performance for natural language processing is judged based on statistical evaluations, however, the truth is perspective, and in this case the qualitative evaluations reveal the real capability of the different architectural strengths and variations. Bayesian Prior definition of different embedding helps in better characterization of the sentences based on the natural language structure related to parts of speech and other semantic level categorization in a form which is machine interpret-able and inherits the characteristics of the Tensor Representation binding and unbinding based on the mutually orthogonality. Our approach has surpassed some of the existing basic works related to image captioning.
Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization

Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones -- and a generator's policy to produce trajectories that can fool this discriminator. This alternated optimization is known to be delicate in practice since it compounds unstable adversarial training with brittle and sample-inefficient reinforcement learning. We propose to remove the burden of the policy optimization steps by leveraging a novel discriminator formulation. Specifically, our discriminator is explicitly conditioned on two policies: the one from the previous generator's iteration and a learnable policy. When optimized, this discriminator directly learns the optimal generator's policy. Consequently, our discriminator's update solves the generator's optimization problem for free: learning a policy that imitates the expert does not require an additional optimization loop. This formulation effectively cuts by half the implementation and computational burden of Adversarial Imitation Learning algorithms by removing the Reinforcement Learning phase altogether. We show on a variety of tasks that our simpler approach is competitive to prevalent Imitation Learning methods.
Bayesian Deep Ensembles via the Neural Tangent Kernel

We explore the link between deep ensembles and Gaussian processes (GPs) through the lens of the Neural Tangent Kernel (NTK): a recent development in understanding the training dynamics of wide neural networks (NNs). Previous work has shown that even in the infinite width limit, when NNs become GPs, there is no GP posterior interpretation to a deep ensemble trained with squared error loss. We introduce a simple modification to standard deep ensembles training, through addition of a computationally-tractable, randomised and untrainable function to each ensemble member, that enables a posterior interpretation in the infinite width limit. When ensembled together, our trained NNs give an approximation to a posterior predictive distribution, and we prove that our Bayesian deep ensembles make more conservative predictions than standard deep ensembles in the infinite width limit. Finally, using finite width NNs we demonstrate that our Bayesian deep ensembles faithfully emulate the analytic posterior predictive when available, and can outperform standard deep ensembles in various out-of-distribution settings, for both regression and classification tasks.
How does Weight Correlation Affect Generalisation Ability of Deep Neural Networks

This paper studies the novel concept of weight correlation in deep neural networks and discusses its impact on the networks' generalisation ability. For fully-connected layers, the weight correlation is defined as the average cosine similarity between weight vectors of neurons, and for convolutional layers, the weight correlation is defined as the cosine similarity between filter matrices. Theoretically, we show that, weight correlation can, and should, be incorporated into the PAC Bayesian framework for the generalisation of neural networks, and the resulting generalisation bound is monotonic with respect to the weight correlation. We formulate a new complexity measure, which lifts the PAC Bayes measure with weight correlation, and experimentally confirm that it is able to rank the generalisation errors of a set of networks more precisely than existing measures. More importantly, we develop a new regulariser for training, and provide extensive experiments that show that the generalisation error can be greatly reduced with our novel approach.
A Joint Model for Aspect-Category Sentiment Analysis with Shared Sentiment Prediction Layer

Aspect-category sentiment analysis (ACSA) aims to predict the aspect categories mentioned in texts and their corresponding sentiment polarities. Some joint models have been proposed to address this task. Given a text, these joint models detect all the aspect categories mentioned in the text and predict the sentiment polarities toward them at once. Although these joint models obtain promising performances, they train separate parameters for each aspect category and therefore suffer from data deficiency of some aspect categories. To solve this problem, we propose a novel joint model which contains a shared sentiment prediction layer. The shared sentiment prediction layer transfers sentiment knowledge between aspect categories and alleviates the problem caused by data deficiency. Experiments conducted on SemEval-2016 Datasets demonstrate the effectiveness of our model.
Spoken notifications in smart environments using Croatian language

Speech technologies have advanced significantly in the last decade, mostly due to rise in available computing power combined with novel approaches to natural language processing. As a result, speech-enabled systems have become popular commercial products, successfully integrated with various environments. However, this can be stated for English and a few other ?big? languages. From the perspective of a minority language, such as Croatian, there are many challenges ahead to achieve comparable results. In this paper, we propose a model for natural language generation and speech synthesis in a smart environment using Croatian language. The model is evaluated on 27 users to estimate the quality of user experience. The evaluation goal was to determine what users perceive to be more important ? generated speech quality or grammatical correctness of the spoken content. It is shown that most users perceived grammatically correct spoken texts as being of the highest quality.
Discourse of advertising

It is not an overstatement to say that these days we are saturated with advertising. We do not even need to enter a commercial space such as a shopping mall or flick through a glossy magazine where we can reasonably expect some advertising practices taking place; public spaces that traditionally had nothing to do with commercial activities such as university buildings, clinics and hospitals all display adverts of many kinds. Every time we do searches on Google, read an online newspaper, or open a social media site, we are immediately bombarded with adverts or branded content of some sort that try to attract our attention and make us click, like, share or download something. In times of late capitalism, mass production and information overload, advertising has become the key marketing tool for attracting attention and converting non-consumers to consumers, making it an all-pervasive form of discourse. Although advertising has a well-defined and easily recognizable purpose (generating revenues), it comes in a variety of discursive, generic and semiotic manifestations. This makes advertising an important and fertile but also challenging ground for a corpus-based (critical) discourse analysis. This chapter reviews contributions of corpus-based research to our understanding of the discourse of advertising. It begins by a brief overview of some of the major developments in advertising including some of its controversial practices. Subsequently, research studies that have used a corpus approach to investigating aspects of advertising are reviewed. This is extended by a focal analysis, which demonstrates how corpus tools can be effectively adopted to explore discursive practices of new forms of digital advertising. The focus is on native ads. The chapter concludes with observations regarding opportunities and limitations of a corpus-based discourse analysis to study advertising and discusses avenues for future research.
Estimating Each Other’s Memory Biases in Dialogue

Conversational memory is subject to a number of biases. For instances, references which were reused during dialogue are remembered better than non-reused references. Two experiments examined whether speakers are aware that they are subject to such biases and whether they use information about reference origin (i.e., information about who said what) to determine which references are remembered better by their partner. Pairs of participants performed a map task followed by a questionnaire that assessed each participant’s content memory as well as each participant’s estimation of his or her partner’s memory. In Experiment 1, the participants were unaware that they would perform a memory test after the map task, whereas the participants were mutually aware of the upcoming memory test in Experiment 2. The results revealed that participants did know that their partner was subject to memory biases and that their estimation of these biases was mainly accurate. The results prevented us from drawing any conclusions about the role of origin memory in this process. These results have important implications for subsequent dialogic partner-adaptation.
Finding Non-uniform Quantization Schemes Using Multi-task Gaussian Processes

We propose a novel method for neural network quantization that casts the neural architecture search problem as one of hyperparameter search to find non-uniform bit distributions throughout the layers of a CNN. We perform the search assuming a Multi-Task Gaussian Processes prior, which splits the problem to multiple tasks, each corresponding to different number of training epochs, and explore the space by sampling those configurations that yield maximum information. We then show that with significantly lower precision in the last layers we achieve a minimal loss of accuracy with appreciable memory savings. We test our findings on the CIFAR10 and ImageNet datasets using the VGG, ResNet and GoogLeNet architectures.
A Symbolic Model for Systematically Analyzing TEE-Based Protocols

Trusted Execution Environment (TEE) has been widely used as an approach to provide an isolated storage and computation environment for various protocols, and thus security features of TEE determine how to design these protocols. In practice, however, new TEE-based protocols are often designed empirically, and a lack of comprehensive analysis against real threat models easily results in vulnerabilities and attacks. Unlike most past work focusing on communication channels or secure enclaves, we present a formal model for TEE-based protocols, which includes a detailed threat model taking into account attacks from both network and TEE-based platforms together with a scalable multiset-rewriting modelling framework instantiated by Tamarin. Based on the proposed threat model and formalism, we use Tamarin to systematically and automatically analyze related offline and web-based protocols considering all combination of threats. The results and comparison highlight the protocols’ advantages and weaknesses inherited from TEE-based platforms. Moreover, we also capture some vulnerabilities that are difficult to be found under the traditional threat model and propose corresponding fixes.
Additional Learning on Object Detection: A Novel Approach in Pornography Classification

In this paper, we proposed a new approach for pornographic classification by recognizing sensitive objects on images. To handle the misdetection and wrong judgment, a novel training strategy named additional learning was developed to help object detection model learns from mistakes, therefore increasing the method performance. Furthermore, a separate SVM classifier was trained to classify pornography and benign images from sexual object detected using Mask R-CNN model. Benchmarked by the NPDI-800 dataset, our proposed method achieved an accuracy of 84.625% and 90.125%, before and after applying additional learning strategy respectively. Besides, our proposed model also improves the false positive rate from 22.16% to 3.56% in our manually collected dataset.
DemodNet: Learning Soft Demodulation from Hard Information Using Convolutional Neural Network

Soft demodulation is a basic module of traditional communication receivers. It converts received symbols into soft bits, that is, log likelihood ratios (LLRs). However, in the non-ideal additive white Gaussian noise (AWGN) channel, it is difficult to accurately calculate the LLR. In this paper, we propose a demodulator, DemodNet, based on a fully convolutional neural network with variable input and output length. We use hard bit information to train the DemodNet, and we propose log probability ratio (LPR) based on the output layer of the trained DemodNet to realize soft demodulation. The simulation results show that under the AWGN channel, the performance of both hard demodulation and soft demodulation of DemodNet is very close to the traditional methods. In three non-ideal channel scenarios, i.e., the presence of frequency deviation, additive generalized Gaussian noise (AGGN) channel, and Rayleigh fading channel, the performance of channel decoding using the soft information LPR obtained by DemodNet is better than the performance of decoding using the exact LLR calculated under the ideal AWGN assumption.