Reviewerly/exHarmony huggingface dataset:
Model of Human Actions Recognition Based on 2D Kernel

Abstract Action Recognition is a domain that gains interest along with the development of specific motion capture equipment, hardware and power of processing. Its many applications in domains such as national security and behavior analysis make it even more popular among the scientific community, especially considering the ascending trend of machine learning methods. Nowadays approaches necessary to solve real life problems through human actions recognition became more interesting. To solve this problem are mainly two approaches when attempting to build a classifier, either using RGB images or sensor data, or where possible a combination of these two. Both methods have advantages and disadvantages and domains of utilization in real life problems, solvable through actions recognition. Using RGB input makes it possible to adopt a classifier on almost any infrastructure without specialized equipment, whereas combining video with sensor data provides a higher accuracy, albeit at a higher cost. Neural networks and especially convolutional neural networks are the starting point for human action recognition. By their nature, they can recognize very well spatial and temporal features, making them ideal for RGB images or sequences of RGB images. In the present paper is proposed the convolutional neural network architecture based on 2D kernels. Its structure, along with metrics measuring the performance, advantages and disadvantages are here illustrated. This solution based on 2D convolutions is fast, but has lower performance compared to other known solutions. The main problem when dealing with videos is the context extraction from a sequence of frames. Video classification using 2D Convolutional Layers is realized either by the most significant frame or by frame to frame, applying a probability distribution over the partial classes to obtain the final prediction. To classify actions, especially when differences between them are subtle, and consists of only a small part of the overall image is difficult. When classifying via the key frames, the total accuracy obtained is around 10%. The other approach, classifying each frame individually, proved to be too computationally expensive with negligible gains.
Improving Fairness and Privacy in Selection Problems

Supervised learning models have been increasingly used for making decisions about individuals in applications such as hiring, lending, and college admission. These models may inherit pre-existing biases from training datasets and discriminate against protected attributes (e.g., race or gender). In addition to unfairness, privacy concerns also arise when the use of models reveals sensitive personal information. Among various privacy notions, differential privacy has become popular in recent years. In this work, we study the possibility of using a differentially private exponential mechanism as a post-processing step to improve both fairness and privacy of supervised learning models. Unlike many existing works, we consider a scenario where a supervised model is used to select a limited number of applicants as the number of available positions is limited. This assumption is well-suited for various scenarios, such as job application and college admission. We use ``equal opportunity'' as the fairness notion and show that the exponential mechanisms can make the decision-making process perfectly fair. Moreover, the experiments on real-world datasets show that the exponential mechanism can improve both privacy and fairness, with a slight decrease in accuracy compared to the model without post-processing.
Introducing the condition system

In the previous chapter, we described three programming concepts which we will now start to utilize to describe and build the components of our condition system. We will apply these concepts to a simple code example that we will extend over time, some of which we'll tell in the form of the story of characters named Tom, Kate, and Mark.
Arabic Speech Classification Method Based on Padding and Deep Learning Neural Network

Deep learning convolution neural network has been widely used to recognize or classify voice. Various techniques have been used together with convolution neural network to prepare voice data before the training process in developing the classification model. However, not all model can produce good classification accuracy as there are many types of voice or speech. Classification of Arabic alphabet pronunciation is a one of the types of voice and accurate pronunciation is required in the learning of the Qur’an reading. Thus, the technique to process the pronunciation and training of the processed data requires specific approach. To overcome this issue, a method based on padding and deep learning convolution neural network is proposed to evaluate the pronunciation of the Arabic alphabet. Voice data from six school children are recorded and used to test the performance of the proposed method. The padding technique has been used to augment the voice data before feeding the data to the CNN structure to developed the classification model. In addition, three other feature extraction techniques have been introduced to enable the comparison of the proposed method which employs padding technique. The performance of the proposed method with padding technique is at par with the spectrogram but better than mel-spectrogram and mel-frequency cepstral coefficients. Results also show that the proposed method was able to distinguish the Arabic alphabets that are difficult to pronounce. The proposed method with padding technique may be extended to address other voice pronunciation ability other than the Arabic alphabets.
Understanding Execution Environment of File-Manipulation Scripts by Extracting Pre-Conditions

File manipulation scripts are widely used in software projects to operate the file system at run time. Due to the emergence of DevOps practices in software industry, developers also use longer and more complicated file manipulations in their continuous integration and deployment scripts to automate software build, testing, and deployment in different environment configurations. A major challenge on understanding these scripts is that they make lots of implicit assumptions on the file system they are executed on. Such assumptions are rarely documented and often do not hold when a script is moved to another execution environment. In this paper, we propose a static-analysis-based technique that statically infer the directory tree pre-condition of the file system required to execute a file manipulation script. We evaluated our analysis on 58 docker files and the experiment shows that our technique is able to generate directory tree preconditions on real world scripts efficiently.
A Semi-Automatic System of Web Videos Annotation and Retrieval: Application to Events Detection in Soccer Domain

Annotations and retrieval of soccer videos on the web is a challenging task that concerns human lives including sports. In this paper, we propose a novel approach based on deep learning and ontology formalism to detect objects and to recognise events in soccer videos. To overcome the semantic gap between low and high level semantic annotation of videos, we use a deep neural network to extract low-level features through a complete method called mask R-CNN based ResNet-101 architecture as a backbone. Then, we create and populate soccer ontology in accordance to the output predictions of the mask R-CNN. We then create a smart system able to learn how to detect objects and to infer events in soccer videos. To validate our approach, we experimented on 40 soccer videos of FIFA World Cup 2018 downloaded from YouTube and we compare the obtained results with those of the state of the art.
Multi-Scale PIIFD for Registration of Multi-Source Remote Sensing Images

This paper aims at providing multi-source remote sensing images registered in geometric space for image fusion. Focusing on the characteristics and differences of multi-source remote sensing images, a feature-based registration algorithm is implemented. The key technologies include image scale-space for implementing multi-scale properties, Harris corner detection for keypoints extraction, and partial intensity invariant feature descriptor (PIIFD) for keypoints description. Eventually, a multi-scale Harris-PIIFD image registration algorithm framework is proposed. The experimental results of fifteen sets of representative real data show that the algorithm has excellent, stable performance in multi-source remote sensing image registration, and can achieve accurate spatial alignment, which has strong practical application value and certain generalization ability.
Analysis of Complex Network Measures for Multi-label Classification

Most multi-label learning (MLL) techniques perform classification by analyzing only the physical features of the data, which means they are unable to consider high-level features, such as structural and topological ones. Consequently, they have trouble to detect the semantic meaning of the data (e.g., formation pattern). To handle this problem, a high-level framework has been recently proposed to the MLL task, in which the high-level features are extracted using the analysis of complex network measures. In this paper, we extend that work by evaluating different combinations of four complex networks measures, namely clustering coefficient, assortativity, average degree and average path length. Experiments conducted over seven real-world data sets showed that the low-level techniques often can have their predictive performance improved after being combined with high-level ones, and also demonstrated that there is no a unique measure that provides the best results, i.e., different problems may ask for different network properties in order to have their high-level patterns efficiently detected.
Exploring the Limits of Vanilla CNN Architectures for Fine-Grained Vision-Based Vehicle Classification

The detection and classification of vehicles by suitable monitoring systems is an integral part of Intelligent Transportation Systems (ITS). We report results of an ongoing research project on fine-grained vehicle classification based on images acquired from roadside and overhead based video cameras. In a previous work [1] a dataset of overall 100,000 sample images from 36 fine-grained vehicle classes has already been presented. These images were acquired from roadside based cameras and results for the classification accuracy obtained with state-of-the-art CNNs (convolutional neural networks) allowed to fulfil the challenging traffic norm TLS 8+1 A1. Here, in extension to this work, cameras in overhead perspective were used to avoid the problem of occlusion (i.e., a larger vehicle completely occluding a smaller one), which currently limits the roadside perspective to two-lane roads (with one camera per lane). Therefore, the original dataset was expanded with a new set of close to 100,000 images now taken in overhead perspective and representing the same 36 fine-grained vehicle classes. While keeping all model and hyperparameters identical (size of training and test set, resolution, CNN architecture, …) in overhead perspective a considerable drop in the classification accuracy was observed with respect to the roadside perspective. Analysis of the confusion matrix reveals that important details of the vehicles, which are essential for the distinction among certain classes, are not sufficiently well represented in the CNN in overhead position. These results seem to indicate, that standard CNNs come to their limits for the present task of fine-grained vehicle classification and other, part-based approaches are required to solve this problem.
Coupled Gradient Estimators for Discrete Latent Variables

Training models with discrete latent variables is challenging due to the high variance of unbiased gradient estimators. While low-variance reparameterization gradients of a continuous relaxation can provide an effective solution, a continuous relaxation is not always available or tractable. Dong et al. (2020) and Yin et al. (2020) introduced a performant estimator that does not rely on continuous relaxations; however, it is limited to binary random variables. We introduce a novel derivation of their estimator based on importance sampling and statistical couplings, which we extend to the categorical setting. Motivated by the construction of a stick-breaking coupling, we introduce gradient estimators based on reparameterizing categorical variables as sequences of binary variables and Rao-Blackwellization. In systematic experiments, we show that our proposed categorical gradient estimators provide state-of-the-art performance, whereas even with additional Rao-Blackwellization, previous estimators (Yin et al., 2019) underperform a simpler REINFORCE with a leave-one-out-baseline estimator (Kool et al., 2019).
Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles

When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance. For safe deployment, it is essential to estimate the accuracy of the pre-trained model on the test data. However, the labels for the test inputs are usually not immediately available in practice, and obtaining them can be expensive. This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs. In this paper, we propose a principled and practically effective framework that simultaneously addresses the two tasks. The proposed framework iteratively learns an ensemble of models to identify mis-classified data points and performs self-training to improve the ensemble with the identified points. Theoretical analysis demonstrates that our framework enjoys provable guarantees for both accuracy estimation and error detection under mild conditions readily satisfied by practical deep learning models. Along with the framework, we proposed and experimented with two instantiations and achieved state-of-the-art results on 59 tasks. For example, on iWildCam, one instantiation reduces the estimation error for unsupervised accuracy estimation by at least 70% and improves the F1 score for error detection by at least 4.7% compared to existing methods.
Learning with invariances in random features and kernel models

A number of machine learning tasks entail a high degree of invariance: the data distribution does not change if we act on the data with a certain group of transformations. For instance, labels of images are invariant under translations of the images. Certain neural network architectures -- for instance, convolutional networks -- are believed to owe their success to the fact that they exploit such invariance properties. With the objective of quantifying the gain achieved by invariant architectures, we introduce two classes of models: invariant random features and invariant kernel methods. The latter includes, as a special case, the neural tangent kernel for convolutional networks with global average pooling. We consider uniform covariates distributions on the sphere and hypercube and a general invariant target function. We characterize the test error of invariant methods in a high-dimensional regime in which the sample size and number of hidden units scale as polynomials in the dimension, for a class of groups that we call `degeneracy $\alpha$', with $\alpha \leq 1$. We show that exploiting invariance in the architecture saves a $d^\alpha$ factor ($d$ stands for the dimension) in sample size and number of hidden units to achieve the same test error as for unstructured architectures. Finally, we show that output symmetrization of an unstructured kernel estimator does not give a significant statistical improvement; on the other hand, data augmentation with an unstructured kernel estimator is equivalent to an invariant kernel estimator and enjoys the same improvement in statistical efficiency.
MDP Playground: A Design and Debug Testbed for Reinforcement Learning

We present \emph{MDP Playground}, an efficient testbed for Reinforcement Learning (RL) agents with \textit{orthogonal} dimensions that can be controlled independently to challenge agents in different ways and obtain varying degrees of hardness in generated environments. We consider and allow control over a wide variety of dimensions, including \textit{delayed rewards}, \textit{rewardable sequences}, \textit{density of rewards}, \textit{stochasticity}, \textit{image representations}, \textit{irrelevant features}, \textit{time unit}, \textit{action range} and more. We define a parameterised collection of fast-to-run toy environments in \textit{OpenAI Gym} by varying these dimensions and propose to use these for the initial design and development of agents. We also provide wrappers that inject these dimensions into complex environments from \textit{Atari} and \textit{Mujoco} to allow for evaluating agent robustness. We further provide various example use-cases and instructions on how to use \textit{MDP Playground} to design and debug agents. We believe that \textit{MDP Playground} is a valuable testbed for researchers designing new, adaptive and intelligent RL agents and those wanting to unit test their agents.
Tourism and Contact Tracing Apps in the COVID-19 Era

The COVID-19 pandemic brought major changes to humanity in a very short period of time. Things that had been established for years no longer apply and a "new normal" has been abruptly introduced into people's lives. At a time when the global tourism industry is struggling to restart, this paper looks at how tourism destinations around the world could benefit by adopting contact tracing applications. The authors argue for a contact tracing application implementation in tourism and propose possible improvements to the Apple/Google implementation of the Exposure Notification System. These improvements include a notification feedback mechanism that will potentially improve the validity of future notifications and a built-in global COVID-19 information feature. Findings acquired through a survey conducted in Greece are analyzed with the aim to investigate whether a contact tracing app is considered useful by the public, tourism destinations and local health authorities.
A New Perspective on Learning Context-Specific Independence

Local structure such as context-specific independence (CSI) has received much attention in the probabilistic graphical model (PGM) literature, as it facilitates the modeling of large complex systems, as well as for reasoning with them. In this paper, we provide a new perspective on how to learn CSIs from data. We propose to first learn a functional and parameterized representation of a conditional probability table (CPT), such as a neural network. Next, we quantize this continuous function, into an arithmetic circuit representation that facilitates efficient inference. In the first step, we can leverage the many powerful tools that have been developed in the machine learning literature. In the second step, we exploit more recently-developed analytic tools from explainable AI, for the purposes of learning CSIs. Finally, we contrast our approach, empirically and conceptually, with more traditional variable-splitting approaches, that search for CSIs more explicitly.
Social Network Influencers’ Data Augmenting Recommender Systems

The ever-increasing use of social network sites and the availability of Internet services have introduced opportunities for users to communicate, interact, and connect with one another. Social network sites can identify the similarity between users in order to recommend new friends to a user. Therefore, users might not only communicate with their actual offline friends, but also they are motivated to communicate with strangers and friends of their actual friends. This paper introduces a model that integrates and incorporates social network influencers’ data for augmenting recommender systems. This model is developed based on three main techniques. The first one is the contentbased technique used to recommend items to an active user based on his/her previously collected data or interests. The second technique is a Bayesian classifier used to learn an active user’s profile and to determine social network influencers who can contribute to augment the quality of recommendations. The third technique is used to identify social media influencers. This model can be generalized to other domains that collect a side information from external sources.
Copy that! Editing Sequences by Copying Spans

Neural sequence-to-sequence models are finding increasing use in editing of documents, for example in correcting a text document or repairing source code. In this paper, we argue that common seq2seq models (with a facility to copy single tokens) are not a natural fit for such tasks, as they have to explicitly copy each unchanged token. We present an extension of seq2seq models capable of copying entire spans of the input to the output in one step, greatly reducing the number of decisions required during inference. This extension means that there are now many ways of generating the same output, which we handle by deriving a new objective for training and a variation of beam search for inference that explicitly handles this problem. In our experiments on a range of editing tasks of natural language and source code, we show that our new model consistently outperforms simpler baselines.
Technological modality to influence persuasive and argumentative vocabulary for effective communication with reference to selected TED talk videos

English as the Second language learning recently gained attention in the field of research. The ESL (English as Second Language) learners need vocabulary enhancement and fluency for proficiency of the language which can be achieved through training. By learning and practicing a language with enhanced vocabulary will increase the vocabulary. TED (Technology, Entertainment, and Design) talks are a world’s biggest digital platform for public speaking. Vocabulary elements given in the TED talks can be defined as a lexicon of language which plays a significant role in communication. The aim of this research conveys the significant role of TED talk videos’ speaker and its influence towards its audiences. This can be achieved only by practical use of vocabulary by the ESL and EFL learners. In this research work, Persuasive and Argumentative vocabulary in the transcript of random 25 TED talk videos with time frame of 0-6mins, and sorted by ‘newest’ tab are analyzed. Also, in-depth analyses of both Persuasive and Augmentative keywords used and its frequencies are listed out from the 25 videos. This research significantly concludes that for effective communication, the learner has to be proficient in vocabulary acquisition.
RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks

Segmenting unseen object instances in cluttered environments is an important capability that robots need when functioning in unstructured environments. While previous methods have exhibited promising results, they still tend to provide incorrect results in highly cluttered scenes. We postulate that a network architecture that encodes relations between objects at a high-level can be beneficial. Thus, in this work, we propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks. We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the perturbed segmentations. Our proposed method is orthogonal to previous works and achieves state-of-the-art performance when combined with them. We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes. Code, models, and video can be found at this https URL .