OpenRaiser/Intern-Atlas huggingface dataset:
Article citation study: Context enhanced citation sentiment detection
Venue: arXiv cs.CL 2020
Year: 2020

Citation sentimet analysis is one of the little studied tasks for scientometric analysis. For citation analysis, we developed eight datasets comprising citation sentences, which are manually annotated by us into three sentiment polarities viz. positive, negative, and neutral. Among eight datasets, three were developed by considering the whole context of citations. Furthermore, we proposed an ensembled feature engineering method comprising word embeddings obtained for texts, parts-of-speech tags, and dependency relationships together. Ensembled features were considered as input to deep learning based approaches for citation sentiment classification, which is in turn compared with Bag-of-Words approach. Experimental results demonstrate that deep learning is useful for higher number of samples, whereas support vector machine is the winner for smaller number of samples. Moreover, context-based samples are proved to be more effective than context-less samples for citation sentiment analysis.
Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures
Venue: arXiv cs.DC 2020
Year: 2020

During the last two years, the goal of many researchers has been to squeeze the last bit of performance out of HPC system for AI tasks. Often this discussion is held in the context of how fast ResNet50 can be trained. Unfortunately, ResNet50 is no longer a representative workload in 2020. Thus, we focus on Recommender Systems which account for most of the AI cycles in cloud computing centers. More specifically, we focus on Facebook's DLRM benchmark. By enabling it to run on latest CPU hardware and software tailored for HPC, we are able to achieve more than two-orders of magnitude improvement in performance (110x) on a single socket compared to the reference CPU implementation, and high scaling efficiency up to 64 sockets, while fitting ultra-large datasets. This paper discusses the optimization techniques for the various operators in DLRM and which component of the systems are stressed by these different operators. The presented techniques are applicable to a broader set of DL workloads that pose the same scaling challenges/characteristics as DLRM.
A Simple Semi-Supervised Learning Framework for Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Semi-supervised learning (SSL) has a potential to improve the predictive performance of machine learning models using unlabeled data. Although there has been remarkable recent progress, the scope of demonstration in SSL has mainly been on image classification tasks. In this paper, we propose STAC, a simple yet effective SSL framework for visual object detection along with a data augmentation strategy. STAC deploys highly confident pseudo labels of localized objects from an unlabeled image and updates the model by enforcing consistency via strong augmentations. We propose experimental protocols to evaluate the performance of semi-supervised object detection using MS-COCO and show the efficacy of STAC on both MS-COCO and VOC07. On VOC07, STAC improves the AP$^{0.5}$ from $76.30$ to $79.08$; on MS-COCO, STAC demonstrates $2{\times}$ higher data efficiency by achieving 24.38 mAP using only 5\% labeled data than supervised baseline that marks 23.86\% using 10\% labeled data. The code is available at https://github.com/google-research/ssl_detection/.
Minimal Linear Codes From Weakly Regular Plateaued Balanced Functions
Venue: arXiv cs.IT 2020
Year: 2020

Linear codes have diverse applications in secret sharing schemes, secure two-party computation, association schemes, strongly regular graphs, authentication codes and communication. There are a large number of linear codes with few weights in the literature, but a little of them are minimal. In this paper, we are using for the first time weakly regular plateaued balanced functions over the finite fields of odd characteristic in the second generic construction method of linear codes. The main results of this paper are stated below. We first construct several three-weight and four-weight linear codes with flexible parameters from weakly regular plateaued balanced functions. It is worth noting that the (almost) optimal codes may be obtained from these functions. We next observe that all codes obtained in this paper are minimal, thereby they can be directly employed to construct secret sharing schemes with high democracy. Finally, the democratic secret sharing schemes are obtained from the dual codes of our minimal codes.
Improved Algorithms for Solving Polynomial Systems over GF(2) by Multiple Parity-Counting
Venue: arXiv cs.DS 2020
Year: 2020

We consider the problem of finding a solution to a multivariate polynomial equation system of degree $d$ in $n$ variables over $\mathbb{F}_2$. For $d=2$, the best-known algorithm for the problem is by Bardet et al. [J. Complexity, 2013] and was shown to run in time $O(2^{0.792n})$ under assumptions that were experimentally found to hold for random equation systems. The best-known worst-case algorithm for the problem is due to Björklund et al. [ICALP'19]. It runs in time $O(2^{0.804n})$ for $d = 2$ and $O(2^{(1 - 1/(2.7d))n})$ for $d > 2$. In this paper, we devise a worst-case algorithm that improves the one by Björklund et al. It runs in time $O(2^{0.6943n})$ (or $O(1.6181^n)$) for $d = 2$ and $O(2^{(1 - 1/(2d))n})$ for $d > 2$. Our algorithm thus outperforms all known worst-case algorithms, as well as ones analyzed for random equation systems. We also devise a second algorithm that outputs all solutions to a polynomial system and has similar complexity to the first (provided that the number of solutions is not too large). A central idea in the work of Björklund et al. was to reduce the problem of finding a solution to a polynomial system over $\mathbb{F}_2$ to the problem of counting the parity of all solutions. A parity-counting instance was then reduced to many smaller parity-counting instances. Our main observation is that these smaller instances are related and can be solved more efficiently by a new algorithm to a problem which we call multiple parity-counting.
On the Petri Nets with a Single Shared Place and Beyond
Venue: arXiv cs.DS 2020
Year: 2020

Petri nets proved useful to describe various real-world systems, but many of their properties are very hard to check. To alleviate this difficulty, subclasses are often considered. The class of weighted marked graphs with relaxed place constraint (WMG=< for short), in which each place has at most one input and one output, and the larger class of choice-free (CF) nets, in which each place has at most one output, have been extensively studied to this end, with various applications. In this work, we develop new properties related to the fundamental and intractable problems of reachability, liveness and reversibility in weighted Petri nets. We focus mainly on the homogeneous Petri nets with a single shared place (H1S nets for short), which extend the expressiveness of CF nets by allowing one shared place (i.e. a place with at least two outputs and possibly several inputs) under the homogeneity constraint (i.e. all the output weights of the shared place are equal). Indeed, this simple generalization already yields new challenging problems and is expressive enough for modeling existing use-cases, justifying a dedicated study. One of our central results is the first characterization of liveness in a subclass of H1S nets more expressive than WMG=< that is expressed by the infeasibility of an integer linear program (ILP) of polynomial size. This trims down the complexity to co-NP, contrasting with the known EXPSPACE-hardness of liveness in the more general case of weighted Petri nets. In the same subclass, we obtain a new reachability property related to the live markings, which is a variant of the well-known Keller's theorem. Another central result is a new reversibility characterization for the live H1S class, simplifying its checking. Finally, we apply our results to use-cases, highlight their scalability and discuss their extensibility to more expressive classes.
Learning Descriptors Invariance Through Equivalence Relations Within Manifold: A New Approach to Expression Invariant 3D Face Recognition
Venue: arXiv cs.CV 2020
Year: 2020

This paper presents a unique approach for the dichotomy between useful and adverse variations of key-point descriptors, namely the identity and the expression variations in the descriptor (feature) space. The descriptors variations are learned from training examples. Based on the labels of the training data, the equivalence relations among the descriptors are established. Both types of descriptor variations are represented by a graph embedded in the descriptor manifold. The invariant recognition is then conducted as a graph search problem. A heuristic graph search algorithm suitable for the recognition under this setup was devised. The proposed approach was tests on the FRGC v2.0, the Bosphorus and the 3D TEC datasets. It has shown to enhance the recognition performance, under expression variations in particular, by considerable margins.
Integrated Methodology to Cognitive Network Slice Management in Virtualized 5G Networks
Venue: arXiv cs.NI 2020
Year: 2020

Fifth Generation (5G) networks are envisioned to be fully autonomous in accordance to the ETSI-defined Zero touch network and Service Management (ZSM) concept. To this end, purpose-specific Machine Learning (ML) models can be used to manage and control physical as well as virtual network resources in a way that is fully compliant to slice Service Level Agreements (SLAs), while also boosting the revenue of the underlying physical network operator(s). This is because specially designed and trained ML models can be both proactive and very effective against slice management issues that can induce significant SLA penalties or runtime costs. However, reaching that point is very challenging. 5G networks will be highly dynamic and complex, offering a large scale of heterogeneous, sophisticated and resource-demanding 5G services as network slices. This raises a need for a well-defined, generic and step-wise roadmap to designing, building and deploying efficient ML models as collaborative components of what can be defined as Cognitive Network and Slice Management (CNSM) 5G systems. To address this need, we take a use case-driven approach to design and present a novel Integrated Methodology for CNSM in virtualized 5G networks based on a concrete eHealth use case, and elaborate on it to derive a generic approach for 5G slice management use cases. The three fundamental components that comprise our proposed methodology include (i) a 5G Cognitive Workflow model that conditions everything from the design up to the final deployment of ML models; (ii) a Four-stage approach to Cognitive Slice Management with an emphasis on anomaly detection; and (iii) a Proactive Control Scheme for the collaboration of different ML models targeting different slice life-cycle management problems.
Multi-Scale Zero-Order Optimization of Smooth Functions in an RKHS
Venue: arXiv cs.LG 2020
Year: 2020

We aim to optimize a black-box function $f:\mathcal{X} \mapsto \mathbb{R}$ under the assumption that $f$ is Hölder smooth and has bounded norm in the RKHS associated with a given kernel $K$. This problem is known to have an agnostic Gaussian Process (GP) bandit interpretation in which an appropriately constructed GP surrogate model with kernel $K$ is used to obtain an upper confidence bound (UCB) algorithm. In this paper, we propose a new algorithm (\texttt{LP-GP-UCB}) where the usual GP surrogate model is augmented with Local Polynomial (LP) estimators of the Hölder smooth function $f$ to construct a multi-scale UCB guiding the search for the optimizer. We analyze this algorithm and derive high probability bounds on its simple and cumulative regret. We then prove that the elements of many common RKHS are Hölder smooth and obtain the corresponding Hölder smoothness parameters, and hence, specialize our regret bounds for several commonly used kernels. When specialized to the Squared Exponential (SE) kernel, \texttt{LP-GP-UCB} matches the optimal performance, while for the case of Matérn kernels $(K_ν)_{ν>0}$, it results in uniformly tighter regret bounds for all values of the smoothness parameter $ν>0$. Most notably, for certain ranges of $ν$, the algorithm achieves near-optimal bounds on simple and cumulative regrets, matching the algorithm-independent lower bounds up to polylog factors, and thus closing the large gap between the existing upper and lower bounds for these values of $ν$. Additionally, our analysis provides the first explicit regret bounds, in terms of the budget $n$, for the Rational-Quadratic (RQ) and Gamma-Exponential (GE). Finally, experiments with synthetic functions as well as a CNN hyperparameter tuning task demonstrate the practical benefits of our multi-scale partitioning approach over some existing algorithms numerically.
Security of Cloud FPGAs: A Survey
Venue: arXiv cs.CR 2020
Year: 2020

Integrating Field Programmable Gate Arrays (FPGAs) with cloud computing instances is a rapidly emerging trend on commercial cloud computing platforms such as Amazon Web Services (AWS), Huawei cloud, and Alibaba cloud. Cloud FPGAs allow cloud users to build hardware accelerators to speed up the computation in the cloud. However, since the cloud FPGA technology is still in its infancy, the security implications of this integration of FPGAs in the cloud are not clear. In this paper, we survey the emerging field of cloud FPGA security, providing a comprehensive overview of the security issues related to cloud FPGAs, and highlighting future challenges in this research area.
Learning to hash with semantic similarity metrics and empirical KL divergence
Venue: arXiv cs.LG 2020
Year: 2020

Learning to hash is an efficient paradigm for exact and approximate nearest neighbor search from massive databases. Binary hash codes are typically extracted from an image by rounding output features from a CNN, which is trained on a supervised binary similar/ dissimilar task. Drawbacks of this approach are: (i) resulting codes do not necessarily capture semantic similarity of the input data (ii) rounding results in information loss, manifesting as decreased retrieval performance and (iii) Using only class-wise similarity as a target can lead to trivial solutions, simply encoding classifier outputs rather than learning more intricate relations, which is not detected by most performance metrics. We overcome (i) via a novel loss function encouraging the relative hash code distances of learned features to match those derived from their targets. We address (ii) via a differentiable estimate of the KL divergence between network outputs and a binary target distribution, resulting in minimal information loss when the features are rounded to binary. Finally, we resolve (iii) by focusing on a hierarchical precision metric. Efficiency of the methods is demonstrated with semantic image retrieval on the CIFAR-100, ImageNet and Conceptual Captions datasets, using similarities inferred from the WordNet label hierarchy or sentence embeddings.
Towards logical negation for compositional distributional semantics
Venue: arXiv cs.CL 2020
Year: 2020

The categorical compositional distributional model of meaning gives the composition of words into phrases and sentences pride of place. However, it has so far lacked a model of logical negation. This paper gives some steps towards providing this operator, modelling it as a version of projection onto the subspace orthogonal to a word. We give a small demonstration of the operators performance in a sentence entailment task.
A Deep Learning Approach for Automatic Detection of Fake News
Venue: arXiv cs.CL 2020
Year: 2020

Fake news detection is a very prominent and essential task in the field of journalism. This challenging problem is seen so far in the field of politics, but it could be even more challenging when it is to be determined in the multi-domain platform. In this paper, we propose two effective models based on deep learning for solving fake news detection problem in online news contents of multiple domains. We evaluate our techniques on the two recently released datasets, namely FakeNews AMT and Celebrity for fake news detection. The proposed systems yield encouraging performance, outperforming the current handcrafted feature engineering based state-of-the-art system with a significant margin of 3.08% and 9.3% by the two models, respectively. In order to exploit the datasets, available for the related tasks, we perform cross-domain analysis (i.e. model trained on FakeNews AMT and tested on Celebrity and vice versa) to explore the applicability of our systems across the domains.
Embedding-based Scientific Literature Discovery in a Text Editor Application
Venue: arXiv cs.IR 2020
Year: 2020

Each claim in a research paper requires all relevant prior knowledge to be discovered, assimilated, and appropriately cited. However, despite the availability of powerful search engines and sophisticated text editing software, discovering relevant papers and integrating the knowledge into a manuscript remain complex tasks associated with high cognitive load. To define comprehensive search queries requires strong motivation from authors, irrespective of their familiarity with the research field. Moreover, switching between independent applications for literature discovery, bibliography management, reading papers, and writing text burdens authors further and interrupts their creative process. Here, we present a web application that combines text editing and literature discovery in an interactive user interface. The application is equipped with a search engine that couples Boolean keyword filtering with nearest neighbor search over text embeddings, providing a discovery experience tuned to an author's manuscript and his interests. Our application aims to take a step towards more enjoyable and effortless academic writing. The demo of the application (https://SciEditorDemo2020.herokuapp.com/) and a short video tutorial (https://youtu.be/pkdVU60IcRc) are available online.
SimpleMKKM: Simple Multiple Kernel K-means
Venue: arXiv cs.LG 2020
Year: 2020

We propose a simple yet effective multiple kernel clustering algorithm, termed simple multiple kernel k-means (SimpleMKKM). It extends the widely used supervised kernel alignment criterion to multi-kernel clustering. Our criterion is given by an intractable minimization-maximization problem in the kernel coefficient and clustering partition matrix. To optimize it, we re-formulate the problem as a smooth minimization one, which can be solved efficiently using a reduced gradient descent algorithm. We theoretically analyze the performance of SimpleMKKM in terms of its clustering generalization error. Comprehensive experiments on 11 benchmark datasets demonstrate that SimpleMKKM outperforms state of the art multi-kernel clustering alternatives.
GACELA -- A generative adversarial context encoder for long audio inpainting
Venue: arXiv cs.SD 2020
Year: 2020

We introduce GACELA, a generative adversarial network (GAN) designed to restore missing musical audio data with a duration ranging between hundreds of milliseconds to a few seconds, i.e., to perform long-gap audio inpainting. While previous work either addressed shorter gaps or relied on exemplars by copying available information from other signal parts, GACELA addresses the inpainting of long gaps in two aspects. First, it considers various time scales of audio information by relying on five parallel discriminators with increasing resolution of receptive fields. Second, it is conditioned not only on the available information surrounding the gap, i.e., the context, but also on the latent variable of the conditional GAN. This addresses the inherent multi-modality of audio inpainting at such long gaps and provides the option of user-defined inpainting. GACELA was tested in listening tests on music signals of varying complexity and gap durations ranging from 375~ms to 1500~ms. While our subjects were often able to detect the inpaintings, the severity of the artifacts decreased from unacceptable to mildly disturbing. GACELA represents a framework capable to integrate future improvements such as processing of more auditory-related features or more explicit musical features.
A Multi-Dimensional Big Data Storing System for Generated COVID-19 Large-Scale Data using Apache Spark
Venue: arXiv cs.DB 2020
Year: 2020

The ongoing outbreak of coronavirus disease (COVID-19) had burst out in Wuhan China, specifically in December 2019. COVID-19 has caused by a new virus that had not been identified in human previously. This was followed by a widespread and rapid spread of this epidemic throughout the world. Daily, the number of the confirmed cases are increasing rapidly, number of the suspect increases, based on the symptoms that accompany this disease, and unfortunately number of the deaths also increase. Therefore, with these increases in number of cases around the world, it becomes hard to manage all these cases information with different situations; if the patient either injured or suspect with which symptoms that appeared on the patient. Therefore, there is a critical need to construct a multi-dimensional system to store and analyze the generated large-scale data. In this paper, a Comprehensive Storing System for COVID-19 data using Apache Spark (CSS-COVID) is proposed, to handle and manage the problem caused by increasing the number of COVID-19 daily. CSS-COVID helps in decreasing the processing time for querying and storing COVID-19 daily data. CSS-COVID consists of three stages, namely, inserting and indexing, storing, and querying stage. In the inserting stage, data is divided into subsets and then index each subset separately. The storing stage uses set of storing-nodes to store data, while querying stage is responsible for handling the querying processes. Using Apache Spark in CSS-COVID leverages the performance of dealing with large-scale data of the coronavirus disease injured whom increase daily. A set of experiments are applied, using real COVID-19 Datasets, to prove the efficiency of CSS-COVID in indexing large-scale data.
The First French COVID19 Lockdown Twitter Dataset
Venue: arXiv cs.SI 2020
Year: 2020

In this paper, we present a mainly French coronavirus Twitter dataset that we have been continuously collecting since lockdown restrictions have been enacted in France (in March 17, 2020). We offer our datasets and sentiment analysis annotations to the research community at https://github.com/calciu/COVID19-LockdownFr. They have been obtained using high performance computing (HPC) capabilities of our university's datacenter. We think that our contribution can facilitate analysis of online conversation dynamics reflecting people sentiments when facing severe home confinement restrictions determined by the outbreak of this world wide epidemic. We hope that our contribution will help decode shared experience and mood but also test the sensitivity of sentiment measurement instruments and incite the development of new instruments, methods and approaches.