Reviewerly/exHarmony huggingface dataset:
A novel memetic genetic algorithm for solving traveling salesman problem based on multi-parent crossover technique

In the present study, a Novel Memetic Genetic Algorithm (NMGA) is developed to solve the Traveling Salesman Problem (TSP). The proposed NMGA is the combination of Boltzmann probability selection and a multi-parent crossover technique with known random mutation. In the proposed multi-parent crossover parents and common crossing point are selected randomly. After comparing the cost/distance with the adjacent nodes (genes) of participated parents, two offspring’s are produced. To establish the efficiency of the developed algorithm standard benchmarks are solved from TSPLIB against classical genetic algorithm (GA) and the fruitfulness of the proposed algorithm is recognized. Some statistical test has been done and the parameters are studied.
Federated machine learning for multi-domain operations at the tactical edge

The Army is evolving its warfighting concepts to militarily compete, penetrate, dis-integrate, and exploit adversaries as part of a Multi-Domain Operations (MDO) Joint Force. Artificial Intelligence/Machine Learning (AI/ML) is critical to the Armys vision for AI-enabled capabilities to achieve MDO but has significant challenges and risks. The Army faces rapidly changing, never-before-seen situations, where pre-existing training data will quickly become ineffective; tactical training data is noisy, incomplete, and erroneous; and adversaries will employ deception. This is especially challenging at the Tactical Edge that operates in complex urban settings that are dynamic, distributed, resource-constrained, fast-paced, contested, and often physically and virtually isolated. Federated machine learning is collaborative training where training data is not exchanged in order to overcome constraints on training data sharing (policy, security, coalition constraints) and/or insufficient network capacity that are prevalent at the Tactical Edge. We describe the applicability of federated machine learning to MDO using a motivating scenario and identify when it is advantageous to be used. The attributes and design inputs for the deployment of AI/ML (learn-infer-act process), the factors that impact learning-inference processes, and the operational factors impacting the deployment of machine learning are identified. We propose strategies for six AI/ML deployment regimes that are at the intersection of total uncertainty (model and environmental) and the operational timeliness that is required, and map AI/ML techniques to address these challenges and requirements. Scientific research questions that must be answered to fill critical knowledge gaps are identified, and ongoing research approaches to answer them are highlighted.
What Makes Training Multi-Modal Networks Hard?

Consider end-to-end training of a multi-modal vs. a single-modal network on atask with multiple input modalities: the multi-modal network receives moreinformation, so it should match or outperform its single-modal counterpart. Inour experiments, however, we observe the opposite: the best single-modalnetwork always outperforms the multi-modal network. This observation isconsistent across different combinations of modalities and on different tasksand benchmarks. This paper identifies two main causes for this performance drop: first,multi-modal networks are often prone to overfitting due to increased capacity.Second, different modalities overfit and generalize at different rates, sotraining them jointly with a single optimization strategy is sub-optimal. Weaddress these two problems with a technique we call Gradient Blending, whichcomputes an optimal blend of modalities based on their overfitting behavior. Wedemonstrate that Gradient Blending outperforms widely-used baselines foravoiding overfitting and achieves state-of-the-art accuracy on various tasksincluding fine-grained sport classification, human action recognition, andacoustic event detection.
Game Theoretical Analysis on Encrypted Cloud Data Deduplication

Duplicated data storage wastes memory resources and brings extra data-management load and cost to cloud service providers (CSPs). Various feasible schemes to deduplicate encrypted cloud data have been reported. However, their successful deployment in practice depends on whether all system players or stakeholders are willing to accept and execute them in a cooperative way, which was scarcely investigated in the previous literature. In this paper, we employ a noncooperative game to model the interactions in a client-side server-controlled deduplication scheme (S-DEDU) and construct an incentive mechanism based on payment discount to motivate its final acceptance. The experimental results based on a real-world dataset demonstrate the individual rationality, incentive compatibility, profitability, and robustness of our incentive mechanism.
Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks

Tight estimation of the Lipschitz constant for deep neural networks (DNNs) is useful in many applications ranging from robustness certification of classifiers to stability analysis of closed-loop systems with reinforcement learning controllers. Existing methods in the literature for estimating the Lipschitz constant suffer from either lack of accuracy or poor scalability. In this paper, we present a convex optimization framework to compute guaranteed upper bounds on the Lipschitz constant of DNNs both accurately and efficiently. Our main idea is to interpret activation functions as gradients of convex potential functions. Hence, they satisfy certain properties that can be described by quadratic constraints. This particular description allows us to pose the Lipschitz constant estimation problem as a semidefinite program (SDP). The resulting SDP can be adapted to increase either the estimation accuracy (by capturing the interaction between activation functions of different layers) or scalability (by decomposition and parallel implementation). We illustrate the utility of our approach with a variety of experiments on randomly generated networks and on classifiers trained on the MNIST and Iris datasets. In particular, we experimentally demonstrate that our Lipschitz bounds are the most accurate compared to those in the literature. We also study the impact of adversarial training methods on the Lipschitz bounds of the resulting classifiers and show that our bounds can be used to efficiently provide robustness guarantees.
RDMA-driven MongoDB: An approach of RDMA enhanced NoSQL paradigm for large-Scale data processing

With the rapid development of big data and data center networks, NoSQL database has won great popularity for its excellent performance in accelerating the performance of many online and offline big data applications, such as HBase, Cassandra and MongoDB. However, due to massive and frequent Create/Update/Retrieval/Delete (CURD) operations, the traditional TCP/IP protocol stack has difficulty to provide the required request rates and response latency for the large-scale NoSQL system. For example, large-scale data migration or synchronization among multiple clusters in a data center results in competition for network bandwidth with high delay. To mitigate such transmission bottleneck, we propose an approach of RDMA-driven document NoSQL Paradigm’ RDMA_Mongo, based on MongoDB. The performance of CURD operations is enhanced by one-sided Remote Direct Memory Access (RDMA) primitives (such as RDMA Read/Write) without involving the TCP/IP stack or CPU. Evaluation under RDMA-enabled network demonstrates that RDMA_Mongo significantly improves the CURD performance, compared with plain MongoDB. The results show that the average insert throughput increases by approximately 30%, the average delete throughput by over 30%, the update by up to 17% and the query throughput by 15% when facing large-scale data requests.
On the Robustness of the Backdoor-based Watermarking in Deep Neural Networks

Obtaining the state of the art performance of deep learning models imposes a high cost to model generators, due to the tedious data preparation and the substantial processing requirements. To protect the model from unauthorized re-distribution, watermarking approaches have been introduced in the past couple of years. We investigate the robustness and reliability of state-of-the-art deep neural network watermarking schemes. We focus on backdoor-based watermarking and propose two -- a black-box and a white-box -- attacks that remove the watermark. Our black-box attack steals the model and removes the watermark with minimum requirements; it just relies on public unlabeled data and a black-box access to the classification label. It does not need classification confidences or access to the model's sensitive information such as the training data set, the trigger set or the model parameters. The white-box attack, proposes an efficient watermark removal when the parameters of the marked model are available; our white-box attack does not require access to the labeled data or the trigger set and improves the runtime of the black-box attack up to seventeen times. We as well prove the security inadequacy of the backdoor-based watermarking in keeping the watermark undetectable by proposing an attack that detects whether a model contains a watermark. Our attacks show that a recipient of a marked model can remove a backdoor-based watermark with significantly less effort than training a new model and some other techniques are needed to protect against re-distribution by a motivated attacker.
Deep Generative Models for Image Generation: A Practical Comparison Between Variational Autoencoders and Generative Adversarial Networks

Deep Learning models can achieve impressive performance in supervised learning but not for unsupervised one. In image generation problem for example, we have no concrete target vector. Generative models have been proven useful for solving this kind of issues. In this paper, we will compare two types of generative models: Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). We apply those methods to different data sets to point out their differences and see their capabilities and limits as well. We find that, while VAEs are easier and faster to train, their results are in general more blurry than the images generated by GANs. These last are more realistic but noisy.
An Effective Hybrid Approach for Optimising the Learning Process of Multi-layer Neural Networks

Finding the optimal connection weights in a neural network is one of the most challenging tasks in machine learning and pattern recognition. The main disadvantage of conventionally used algorithms such as back-propagation is that they show a tendency of getting trapped in local rather than global optima. To address this, population-based metaheuristic algorithms can be employed. In this paper, we propose a novel approach to optimise the weights of a neural network. Our method integrates an imperialist competitive algorithm, a powerful metaheuristic algorithm, with chaos theory and back-propagation for neural network learning. Experimental results on the three-bit parity problem and several function approximation tasks confirm that our proposed algorithm significantly outperforms several state-of-the-art methods for neural network weight optimisation.
Content Differences in Syntactic and Semantic Representation

Daniel Hershcovich, Omri Abend, Ari Rappoport. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Automatically Generating Psychiatric Case Notes From Digital Transcripts of Doctor-Patient Conversations

Electronic health records (EHRs) are notorious for reducing the face-to-face time with patients while increasing the screen-time for clinicians leading to burnout. This is especially problematic for psychiatry care in which maintaining consistent eye-contact and non-verbal cues are just as important as the spoken words. In this ongoing work, we explore the feasibility of automatically generating psychiatric EHR case notes from digital transcripts of doctor-patient conversation using a two-step approach: (1) predicting semantic topics for segments of transcripts using supervised machine learning, and (2) generating formal text of those segments using natural language processing. Through a series of preliminary experimental results obtained through a collection of synthetic and real-life transcripts, we demonstrate the viability of this approach.
ARMv8 SIKE: Optimized Supersingular Isogeny Key Encapsulation on ARMv8 Processors

In this paper, we present highly-optimized constant-time software libraries for supersingular isogeny key encapsulation (SIKE) protocol on ARMv8 processors. Our optimized hand-crafted assembly libraries provide the most efficient timing results on 64-bit ARM-powered devices. Moreover, the presented libraries can be integrated into any other cryptography primitives targeting the same finite field size. We design a new mixed implementation of field arithmetic on 64-bit ARM processors by exploiting the A64 and Advanced SIMD processing units working in parallel. Using these techniques, we are able to improve the performance of the entire protocol by the factor of $5\times $ compared to optimized C implementations on 64-bit ARM high-performance cores, providing 83-, 124-, and 159-bit quantum-security levels. Furthermore, we compare the performance of our proposed library with the previous highly-optimized ARMv8 assembly library available in the literature. The implementation results illustrate the overall 10% performance improvement in comparison with previous work, highlighting the benefit of using mixed implementation over relatively-large finite field size.
Accounting and Taxation of Cryptoassets

Cryptoassets have attracted significant and growing attention from consumers, regulators and markets globally during recent years.We observe that the current accounting frameworks all over the world, at least at first sight, lack any clear guidance on what accounting treatment may be acceptable for cryptoassets and in particular for cryptocurrencies. The same can be said for taxation, although most countries have issued some basic guidance over the past few years, there is still substantial ambiguity on several important questions about the taxation of cryptoassets and no international approach has been applied
Gradient Descent for One-Hidden-Layer Neural Networks: Polynomial Convergence and SQ Lower Bounds.

We study the complexity of training neural network models with one hidden nonlinear activation layer and an output weighted sum layer. We analyze Gradient Descent applied to learning a bounded target function on $n$ real-valued inputs. We give an agnostic learning guarantee for GD: starting from a randomly initialized network, it converges in mean squared loss to the minimum error (in $2$-norm) of the best approximation of the target function using a polynomial of degree at most $k$. Moreover, for any $k$, the size of the network and number of iterations needed are both bounded by $n^{O(k)}\log(1/\epsilon)$. In particular, this applies to training networks of unbiased sigmoids and ReLUs. We also rigorously explain the empirical finding that gradient descent discovers lower frequency Fourier components before higher frequency components. We complement this result with nearly matching lower bounds in the Statistical Query model. GD fits well in the SQ framework since each training step is determined by an expectation over the input distribution. We show that any SQ algorithm that achieves significant improvement over a constant function with queries of tolerance some inverse polynomial in the input dimensionality $n$ must use $n^{\Omega(k)}$ queries even when the target functions are restricted to a set of $n^{O(k)}$ degree-$k$ polynomials, and the input distribution is uniform over the unit sphere; for this class the information-theoretic lower bound is only $\Theta(k \log n)$. Our approach for both parts is based on spherical harmonics. We view gradient descent as an operator on the space of functions, and study its dynamics. An essential tool is the Funk-Hecke theorem, which explains the eigenfunctions of this operator in the case of the mean squared loss.
Using reinforcement learning to optimize the acceptance threshold of a credit scoring model

This paper aims to study whether the reinforcement learning approach to optimizing the acceptance threshold of a credit score leads to higher profits for the lender compared to the state-of-the-art cost-sensitive optimization approach. We show that static methods, such as the latter do not ensure the optimality of the threshold leading to biased results and significant losses for the firm. We develop a dynamic reinforcement learning system that constantly adapts the threshold in response to live data feedback in order to maximize a company's profits. The developed algorithm is shown to outperform the traditional approach in terms of profits both in various simulated scenarios and using real data from an international consumer credit company.
Sentiment Analysis of Turkish and English Twitter Feeds Using Word2Vec Model

Social media has become an important part of daily life. With twitter, one of the most popular social media services, users express their feelings and thoughts to the whole world using twitter posts. For this reason, twitter feeds have become an important source of sentiment analysis. In this study, the apply of Word2Vec model in the classification of labeled data in English and Turkish Twitter feeds and the effect of getting root on feeds to Word2Vec model are investigated. Our study has two different data sets, English and Turkish. BOW and Word2Vec models were applied to each data set in the case where twitter feeds were not get roots and get roots were extracted. In this study, which is implemented in the Python programming language, the success percentages are compared by applying the scikit-learn classification algorithms, Linear SVM and Logistic Regression.
Isolated Sign Language Recognition with Multi-scale Features using LSTM

Sign language recognition systems are used to convert signs in video streams to text automatically. In this work, an original isolated sign language recognition model is created using Convolutional Neural Networks (CNNs), Feature Pooling Module and Long Short-Term Memory Networks (LSTMs). In the CNN part, a pre-trained VGG-16 model is used identically in two parallel architectures, after adapting its weights to the dataset; in this architecture, the features from color (RGB) and depth streams are extracted in parallel. The extracted features are directed to FPM to generate multi-scale features. The features matrices are reduced to representative feature vectors, using Global Average Pooling (GAP). The features that are obtained from RGB and depth streams are concatenated and passed to the LSTM architecture after instance normalization. We get 93.15% test accuracy on Montalbano Italian sign language dataset using the proposed model; this result is comparable with the recent state-of-the-art methods.
Optimizing the Bit Allocation for Compression of Weights and Activations of Deep Neural Networks

For most real-time implementations of deep artificial neural networks, both weights and intermediate layer activations have to be stored and loaded for processing. Compression of both is often advisable to mitigate the memory bottleneck. In this paper, we propose a bit allocation framework for compressing the weights and activations of deep neural networks. The differentiability of input-output relationships for all network layers allows us to relate the neural network output accuracy to the bit-rate of quantized weights and layer activations. We formulate a Lagrangian optimization framework that finds the optimum joint bit allocation among all intermediate activation layers and weights. Our method obtains excellent results on two deep neural networks, VGG-16 and ResNet-50. Without requiring re-training, it outperforms other state-of-the-art neural network compression methods.
Learning Temporal Pose Estimation from Sparsely-Labeled Videos

Modern approaches for multi-person pose estimation in video require large amounts of dense annotations. However, labeling every frame in a video is costly and labor intensive. To reduce the need for dense annotations, we propose a PoseWarper network that leverages training videos with sparse annotations (every k frames) to learn to perform dense temporal pose propagation and estimation. Given a pair of video frames---a labeled Frame A and an unlabeled Frame B---we train our model to predict human pose in Frame A using the features from Frame B by means of deformable convolutions to implicitly learn the pose warping between A and B. We demonstrate that we can leverage our trained PoseWarper for several applications. First, at inference time we can reverse the application direction of our network in order to propagate pose information from manually annotated frames to unlabeled frames. This makes it possible to generate pose annotations for the entire video given only a few manually-labeled frames. Compared to modern label propagation methods based on optical flow, our warping mechanism is much more compact (6M vs 39M parameters), and also more accurate (88.7% mAP vs 83.8% mAP). We also show that we can improve the accuracy of a pose estimator by training it on an augmented dataset obtained by adding our propagated poses to the original manual labels. Lastly, we can use our PoseWarper to aggregate temporal pose information from neighboring frames during inference. This allows us to obtain state-of-the-art pose detection results on PoseTrack2017 and PoseTrack2018 datasets.
Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks

We present a comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory. Our contributions are twofold: (i) we develop an end-to-end framework to train a binary activated deep neural network, overcoming the fact that binary activation function is non-differentiable; (ii) we provide nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks. Noteworthy, our results are obtained by minimizing the expected loss of an architecture-dependent aggregation of binary activated deep neural networks. The performance of our approach is assessed on a thorough numerical experiment protocol on real-life datasets.
Space and Time Efficient Kernel Density Estimation in High Dimensions

Recently, Charikar and Siminelakis (2017) presented a framework for kernel density estimation in provably sublinear query time, for kernels that possess a certain hashing-based property. However, their data structure requires a significantly increased super-linear storage space, as well as super-linear preprocessing time. These limitations inhibit the practical applicability of their approach on large datasets. In this work, we present an improvement to their framework that retains the same query time, while requiring only linear space and linear preprocessing time. We instantiate our framework with the Laplacian and Exponential kernels, two popular kernels which possess the aforementioned property. Our experiments on various datasets verify that our approach attains accuracy and query time similar to Charikar and Siminelakis (2017), with significantly improved space and preprocessing time.