Reviewerly/exHarmony huggingface dataset:
Image Captioning using Google's Inception-resnet-v2 and Recurrent Neural Network

Given a photograph as input, this paper solves the problem of experiencing a plausible caption of the photograph. The model learns about the correlations between language and images from the provided data-set of labeled images. It proposes a fully automatic approach through a combination of Convolutional Neural Network and a Recurrent Neural Network. The encoder is responsible for understanding the features present in the inputted image that are useful in eventually producing an explanation. The model attempts at producing captions for both the objects and the regions present in the image. Treating language as a big label space, the project generates predictions for the various regions of the image and then stitches them together.
Rivalry and Lack of Blocking Among Italian and German Diminutives in Adult and Child Language

This contribution deals with competition (rivalry) between diminutive suffixations in (Austrian) German and Italian adult language and child speech and the motivation of differences in terms of productivity and degree of morphosemantic transparency/opacity of the respective diminutives. For this purpose productivity, availability and profitability are differentiated, a scale of morphosemantic transparency/opacity is established which ranges from full transparency (degree 1), over slight pragmatic restrictions (degree 2), to total opacity (unless in metalinguistic reflections: degree 9) and fake decompositionality (degree 10). The degrees of transparency/opacity differ for different suffixations, in German also between compound and simplex diminutives. Longitudinal studies of early child language show that child language is more transparent than adult language in types and tokens. In both there is competition between fully transparent suffixations (degree 1) without any pattern blocking or lexical blocking, because such diminutives are mainly formed and used for pragmatic reasons and never for the purpose of lexical enrichment. Since productivity, availability and profitability of diminutive formation is much higher in Italian than in German, there is much more competition between diminutive suffixations in Italian than in German adult language. This difference between the two languages is much less the case in early child language. Possible psycholinguistic consequences are mentioned in the outlook.
T-DLA: An Open-source Deep Learning Accelerator for Ternarized DNN Models on Embedded FPGA

Deep Neural Networks (DNNs) have become promising solutions for data analysis especially for raw data processing from sensors. However, using DNN-based approaches can easily introduce huge demands of computation and memory consumption, which may not be feasible for direct deployment onto the Internet of Thing (IoT) devices, since they have strict constraints on hardware resources, power budgets, response latency, and manufacturing cost. To bring DNNs into IoT devices, embedded FPGA can be one of the most suitable candidates by providing better energy efficiency than GPU and CPU based solutions, and higher flexibility than ASICs. In this paper, we propose a systematic solution to deploy DNNs on embedded FPGAs, which includes a ternarized hardware Deep Learning Accelerator (T-DLA), and a framework for ternary neural network (TNN) training. T-DLA is a highly optimized hardware unit in FPGA specializing in accelerating the TNNs, while the proposed framework can significantly compress the DNN parameters down to two bits with little accuracy drop. Results show that our training framework can compress the DNN up to 14.14x while maintaining nearly the same accuracy compared to the floating point version. By illustrating our proposed design techniques, the T-DLA can deliver up to 0.4TOPS with 2.576W power consumption, showing 873.6x and 5.1x higher energy efficiency (fps/W) on ImageNet with Resnet-18 model comparing to Xeon E5-2630 CPU and Nvidia 1080 Ti GPU. To the best of our knowledge, this is the first instruction-based highly efficient ternary DLA design reported from the literature.
Deep Learning and Data Sampling with Imbalanced Big Data

This study evaluates the use of deep learning and data sampling on a class-imbalanced Big Data problem, i.e. Medicare fraud detection. Medicare offers affordable health insurance to the elderly population and serves more than 15% of the United States population. To increase transparency and help reduce fraud, the Centers for Medicare and Medicaid Services (CMS) have made several data sets publicly available for analysis. Our research group has conducted several studies using CMS data and traditional machine learning algorithms (non-deep learning), but challenges associated with severe class imbalance leave room for improvement. These previous studies serve as baselines as we employ deep neural networks with various data-sampling techniques to determine the efficacy of deep learning in addressing class imbalance. Random over-sampling (ROS), random under-sampling (RUS), and combinations of the two (ROS-RUS) are applied to study how varying levels of class imbalance impact model training and performance. Classwise performance is maximized by identifying optimal decision thresholds, and a strong linear relationship between minority class size and optimal threshold is observed. Results show that ROS significantly outperforms RUS, combining RUS and ROS both maximizes performance and efficiency with a 4× speedup in training time, and the default threshold of 0.5 is never optimal when training data is imbalanced. To the best of our knowledge, this is the first study to provide statistical results comparing ROS, RUS, and ROS-RUS deep learning methods across a range of class distributions. Additional contributions include a unique analysis of thresholding as it relates to the minority class size and state-of-the-art performance on the given fraud detection task.
Trust-PBFT: A PeerTrust-Based Practical Byzantine Consensus Algorithm

Blockchain is becoming increasingly popular, while the performance problems have confused its implementation, especially the limited size of the distributed network. In this paper, we propose Trust-PBFT, a combination of PeerTrust P2P trust calculation model and PBFT consensus algorithm. We first introduce PeerTrust to evaluate the trustworthy of the nodes that qualify as participants of PBFT, which replaces the original that all nodes participate in so that the size of the distributed network can be expanded arbitrarily. Besides, in order to simplify the structure of block storage, particularly the storage of the feedbacks of transaction used to calculate trust value of nodes, we design a tri-chain architecture to store accounts, transactions and feedbacks in three blockchains, respectively. Moreover, due to the introduction of PeerTrust, a weak benefit is to improve the fault tolerance performance efficiently. Finally, we design simulation experiments to evaluate the fault tolerance performance and scalability of Trust-PBFT.
Building a morpho-semantic knowledge graph for Arabic information retrieval

In this paper, we propose to build a morpho-semantic knowledge graph from Arabic vocalized corpora. Our work focuses on classical Arabic as it has not been deeply investigated in related works. We use a tool suite which allows analyzing and disambiguating Arabic texts, taking into account short diacritics to reduce ambiguities. At the morphological level, we combine Ghwanmeh stemmer and MADAMIRA which are adapted to extract a multi-level lexicon from Arabic vocalized corpora. At the semantic level, we infer semantic dependencies between tokens by exploiting contextual knowledge extracted by a concordancer. Both morphological and semantic links are represented through compressed graphs, which are accessed through lazy methods. These graphs are mined using a measure inspired from BM25 to compute one-to-many similarity. Indeed, we propose to evaluate the morpho-semantic Knowledge Graph in the context of Arabic Information Retrieval (IR). Several scenarios of document indexing and query expansion are assessed. That is, we vary indexing units for Arabic IR based on different levels of morphological knowledge, a challenging issue which is not yet resolved in previous research. We also experiment several combinations of morpho-semantic query expansion. This permits to validate our resource and to study its impact on IR based on state-of-the art evaluation metrics.
Forecasting the aggregated output of a large fleet of small behind-the-meter solar photovoltaic sites

Significant growth of behind-the-meter solar Photovoltaic (PV) power generation in recent years is changing the shape of the net demand for electricity from electrical grids. In this work, a framework is proposed to forecast the aggregated power generation of a large fleet of small behind-the-meter solar PV sites. The outputs of those sites are not individually measured and thus, the aggregated output is “invisible” to power system operators. The proposed model uses the available historical power generation data of a very limited number of representative sites in the region, along with Numerical Weather Predictions (NWP) inputs. This way, it is not necessary to constantly monitor all the sites in the region. Fuzzy Arithmetic Wavelet Neural Networks (FAWNN) are used to develop the forecasting engine, providing fuzzy confidence intervals for any desired level, so this methodology can handle various shapes of uncertainties in the input data. The proposed model is validated using actual PV generation data from 6673 sites in California. The simulation results have shown that the proposed approach is capable of forecasting BTM solar PV fleet despite using limited data. The root mean squared error for the forecasts was found to be 3% for the California region.
Single-Server Single-Message Online Private Information Retrieval with Side Information

In many practical settings, the user needs to retrieve information messages from a server in a periodic manner, over multiple rounds of communication. The messages are retrieved one at a time and the identity of future requests are not known to the server. In this paper, we focus on the private information retrieval protocols that ensure that the identities of all the messages retrieved from the server are protected. This scenario can occur in practical settings such as periodic content download from text and multimedia repositories. We refer to this problem of minimizing the rate of data download as online private information retrieval problem.Following the previous line of work by Kadhe et al. we assume that the user knows a subset of M messages in the database as side information. The identities of these M messages are initially unknown to the server. Focusing on scalar-linear settings, we characterize the per-round capacity, i.e., the maximum achievable download rate at each round. In particular, we show that for the setting with K messages stored at the server, the per-round capacity of the scalar-linear setting is C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> = (M + 1)/K for round i = 1 and C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> = (2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i -1</sup> (M + 1))/KM for round i ≥ 2, provided that K/(M + 1) is a power of 2. The key idea≥of our achievability scheme is to combine the data downloaded during the current round and the previous rounds with the original side information messages and use the resulting data as side information for the subsequent rounds.
Deep conditional adaptation networks and label correlation transfer for unsupervised domain adaptation

Unsupervised domain adaptation aims to improve the performance of an unknown target domain by utilizing the knowledge learned from a related source domain. Given that the target label information is unavailable in the unsupervised situation, it is challenging to match the domain distributions and to transfer the source model to target applications. In this paper, a Deep Conditional Adaptation Networks (DCAN) is proposed to address the unsupervised domain adaptation problem. DCAN is implemented based on a deep neural network and attempts to learn domain invariant features based on the Wasserstein distance. A conditional adaptation strategy is presented to reduce the domain distribution discrepancy and to address category mismatch and class prior bias, which are usually ignored in marginal adaptation approaches. Furthermore, we propose a label correlation transfer algorithm to address the unsupervised issues, by generating more effective pseudo target labels based on the underlying cross-domain relationship. A set of comparative experiments were performed on standard domain adaptation benchmarks and the results demonstrate that the proposed DCAN outperforms previous adaptation methods.
Source Code Authorship Attribution Using Hybrid Approach of Program Dependence Graph and Deep Learning Model

Attribution (SCAA) is to find the real author of source code in a corpus.Though, it is a privacy threat to open-source programmers, but, it may be significantly helpful to develop forensic based applications.Such as, ghostwriting detection, copyright dispute settlements, and other code analysis applications.The efficient features extraction is the key challenge for classifying real authors of specific source codes.In this paper, the Program Dependence Graph with Deep Learning (PDGDL) methodology is proposed to identify authors from different programming source codes.First, the PDG is implemented to extract control and data dependencies from source codes.Second, the preprocessing technique is applied to convert PDG features into small instances with frequency details.Third, the Term Frequency Inverse Document Frequency (TFIDF) technique is used to zoom the importance of each PDG feature in source code.Fourth, Synthetic Minority Over-sampling Technique (SMOTE) is applied to tackle the class imbalance problem.Finally, the deep learning algorithm is applied to extract coding styles' features for each programmer and to attribute the real authors.The deep learning algorithm is further fine-tuned with drop out layer, learning error rate, loss and activation function, and dense layers for better accuracy of results.The proposed work is analyzed on 1000 programmers' data, collected from Google Code Jam (GCJ).The dataset contains three different programming languages, i.e., C++, Java, C#.The results are appreciable in outperforming the existing techniques from the perspective of classification accuracy, precision, recall, and f-measure metrics.
An empirical study of security warnings from static application security testing tools

The Open Web Application Security Project (OWASP) defines Static Application Security Testing (SAST) tools as those that can help find security vulnerabilities in the source code or compiled code of software. Such tools detect and classify the vulnerability warnings into one of many types (e.g., input validation and representation). It is well known that these tools produce high numbers of false positive warnings. However, what is not known is if specific types of warnings have a higher predisposition to be false positives or not. Therefore, our goal is to investigate the different types of SAST-produced warnings and their evolution over time to determine if one type of warning is more likely to have false positives than others. To achieve our goal, we carry out a large empirical study where we examine 116 large and popular C++ projects using six different state-of-the-art open source and commercial SAST tools that detect security vulnerabilities. In order to track a piece of code that has been tagged with a warning, we use a new state of the art framework called cregit+ that traces source code lines across different commits. The results demonstrate the potential of using SAST tools as an assessment tool to measure the quality of a product and the possible risks without manually reviewing the warnings. In addition, this work shows that pattern-matching static analysis technique is a very powerful method when combined with other advanced analysis methods.
Gradient Coding Based on Block Designs for Mitigating Adversarial Stragglers

Distributed implementations of gradient-based methods, wherein a server distributes gradient computations across worker machines, suffer from slow running machines, called stragglers. Gradient coding is a coding-theoretic framework to mitigate stragglers by enabling the server to recover the gradient sum in the presence of stragglers. Approximate gradient codes are variants of gradient codes that reduce computation and storage overhead per worker by allowing the server to approximately reconstruct the gradient sum.In this work, our goal is to construct approximate gradient codes that are resilient to stragglers selected by a computationally unbounded adversary. Our motivation for constructing codes to mitigate adversarial stragglers stems from the challenge of tackling stragglers in massive-scale elastic and serverless systems, wherein it is difficult to statistically model stragglers. Towards this end, we propose a class of approximate gradient codes based on balanced incomplete block designs (BIBDs). We show that the approximation error for these codes depends only on the number of stragglers, and thus, adversarial straggler selection has no advantage over random selection. In addition, the proposed codes admit computationally efficient decoding at the server. Next, to characterize fundamental limits of adversarial straggling, we consider the notion of adversarial threshold - the smallest number of workers that an adversary must straggle to inflict certain approximation error. We compute a lower bound on the adversarial threshold, and show that codes based on symmetric BIBDs maximize this lower bound among a wide class of codes, making them excellent candidates for mitigating adversarial stragglers.
Adaptive Single Image Dehazing Using Joint Local-Global Illumination Adjustment

Haze has a serious impact on the outdoor optical imaging systems, and it will result in image blurring, color shift, and saturation reduction. Recently, many single image dehazing algorithms have been proposed for practical applications, such as surveillance. However, since the widely-used global atmospheric light in image dehazing fails to well describe the local illumination differences of images, these algorithms fail to well adapt to scenes with different haze concentrations and lighting conditions. Therefore, this paper proposes an adaptive single image dehazing algorithm using joint local-global illumination adjustment. A local illumination estimation for hazy image is proposed to replace the global atmospheric light constant in the atmospheric scattering model, and it can better adapt to the local differences of image illumination. Correspondingly, the global atmospheric light constant is proposed to be utilized to adaptively compensate the illumination intensity, which may better overcome the dark illumination problem within the dehazed image. The experimental results demonstrate that the proposed algorithm can outperform the state-of-the-art algorithms in terms of not only the dehazing effect but also the adaptability.
Looking back at Labels: A Class based Domain Adaptation Technique

In this paper, we solve the problem of adapting classifiers across domains. We consider the problem of domain adaptation for multi-class classification where we are provided a labeled set of examples in a source dataset and we are provided a target dataset with no supervision. In this setting, we propose an adversarial discriminator based approach. While the approach based on adversarial discriminator has been previously proposed; in this paper, we present an informed adversarial discriminator. Our observation relies on the analysis that shows that if the discriminator has access to all the information available including the class structure present in the source dataset, then it can guide the transformation of features of the target set of classes to a more structure adapted space. Using this formulation, we obtain state-of-the-art results for the standard evaluation on benchmark datasets. We further provide detailed analysis which shows that using all the labeled information results in an improved domain adaptation.
A novel approach inspired by optic nerve characteristics for few-shot occluded face recognition

Although there has been a growing body of work for face recognition, it is still a challenging task for faces under occlusion with limited training samples. In this work, we propose a novel framework to address the problem of few-shot occluded face recognition. In particular, inspired by the human being’s optic nerves characteristics that humans recognize the face under occlusion using contextual information rather than paying attention to the facial parts, we propose an effective feature extraction approach to capture the local and contextual information for face recognition. To enhance the robustness, we further introduce an adaptive fusion method to incorporate multiple features, including the proposed structural element feature, connected-granule labeling feature, and Reinforced Centrosymmetric Local Binary Pattern (RCSLBP). Final recognition is derived from the fusion of all classification results according to our proposed novel fusion method. Experimental results on three popular face image datasets of AR, Extended Yale B, and LFW demonstrate that our method performs better than many existing ones for few-shot face recognition in the presence of occlusion.
Fast Supervised Topic Models for Short Text Emotion Detection

With the development of social network platforms, discussion forums, and question answering websites, a huge number of short messages that typically contain a few words for an individual document are posted by online users. In these short messages, emotions are frequently embedded for communicating opinions, expressing friendship, and promoting influence. It is quite valuable to detect emotions from short messages, but the corresponding task suffers from the sparsity of feature space. In this article, we first generate term groups co-occurring in the same context to enrich the number of features. Then, two basic supervised topic models are proposed to associate emotions with topics accurately. To reduce the time cost of parameter estimation, we further propose an accelerated algorithm for our basic models. Extensive evaluations using three short corpora validate the efficiency and effectiveness of the accelerated models for predicting the emotions of unlabeled documents, in addition to generate the topic-level emotion lexicons.
A Convex Relaxation for Multi-Graph Matching

We present a convex relaxation for the multi-graph matching problem.Our formulation allows for partial pairwise matchings, guarantees cycle consistency, and our objective incorporates both linear and quadratic costs.Moreover, we also present an extension to higher-order costs.In order to solve the convex relaxation we employ a message passing algorithm that optimizes the dual problem.We experimentally compare our algorithm on established benchmark problems from computer vision, as well as on large problems from biological image analysis, the size of which exceed previously investigated multi-graph matching instances.