OpenRaiser/Intern-Atlas huggingface dataset:
Hardness Results for the Synthesis of $b$-bounded Petri Nets (Technical Report)
Venue: arXiv cs.LO 2019
Year: 2019

Synthesis for a type $τ$ of Petri nets is the following search problem: For a transition system $A$, find a Petri net $N$ of type $τ$ whose state graph is isomorphic to $A$, if there is one. To determine the computational complexity of synthesis for types of bounded Petri nets we investigate their corresponding decision version, called feasibility. We show that feasibility is NP-complete for (pure) $b$-bounded P/T-nets if $b\in \mathbb{N}^+$. We extend (pure) $b$-bounded P/T-nets by the additive group $\mathbb{Z}_{b+1}$ of integers modulo $(b+1)$ and show feasibility to be NP-complete for the resulting type. To decide if $A$ has the event state separation property is shown to be NP-complete for (pure) $b$-bounded and group extended (pure) $b$-bounded P/T-nets. Deciding if $A$ has the state separation property is proven to be NP-complete for (pure) $b$-bounded P/T-nets.
Deep Industrial Espionage
Venue: arXiv cs.CV 2019
Year: 2019

The theory of deep learning is now considered largely solved, and is well understood by researchers and influencers alike. To maintain our relevance, we therefore seek to apply our skills to under-explored, lucrative applications of this technology. To this end, we propose and Deep Industrial Espionage, an efficient end-to-end framework for industrial information propagation and productisation. Specifically, given a single image of a product or service, we aim to reverse-engineer, rebrand and distribute a copycat of the product at a profitable price-point to consumers in an emerging market---all within in a single forward pass of a Neural Network. Differently from prior work in machine perception which has been restricted to classifying, detecting and reasoning about object instances, our method offers tangible business value in a wide range of corporate settings. Our approach draws heavily on a promising recent arxiv paper until its original authors' names can no longer be read (we use felt tip pen). We then rephrase the anonymised paper, add the word "novel" to the title, and submit it a prestigious, closed-access espionage journal who assure us that someday, we will be entitled to some fraction of their extortionate readership fees.
Cyberthreat Detection from Twitter using Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

To be prepared against cyberattacks, most organizations resort to security information and event management systems to monitor their infrastructures. These systems depend on the timeliness and relevance of the latest updates, patches and threats provided by cyberthreat intelligence feeds. Open source intelligence platforms, namely social media networks such as Twitter, are capable of aggregating a vast amount of cybersecurity-related sources. To process such information streams, we require scalable and efficient tools capable of identifying and summarizing relevant information for specified assets. This paper presents the processing pipeline of a novel tool that uses deep neural networks to process cybersecurity information received from Twitter. A convolutional neural network identifies tweets containing security-related information relevant to assets in an IT infrastructure. Then, a bidirectional long short-term memory network extracts named entities from these tweets to form a security alert or to fill an indicator of compromise. The proposed pipeline achieves an average 94% true positive rate and 91% true negative rate for the classification task and an average F1-score of 92% for the named entity recognition task, across three case study infrastructures.
The Discrete Fourier Transform for Golden Angle Linogram Sampling
Venue: arXiv cs.GR 2019
Year: 2019

Estimation of the Discrete-Time Fourier Transform (DTFT) at points of a finite domain arises in many imaging applications. A new approach to this task, the Golden Angle Linogram Fourier Domain (GALFD), is presented, together with a computationally fast and accurate tool, named Golden Angle Linogram Evaluation (GALE), for approximating the DTFT at points of a GALFD. A GALFD resembles a Linogram Fourier Domain (LFD), which is efficient and accurate. A limitation of linograms is that embedding an LFD into a larger one requires many extra points, at least doubling the domain's cardinality. The GALFD, on the other hand, allows for incremental inclusion of relatively few data points. Approximation error bounds and floating point operations counts are presented to show that GALE computes accurately and efficiently the DTFT at the points of a GALFD. The ability to extend the data collection in small increments is beneficial in applications such as Magnetic Resonance Imaging. Experiments for simulated and for real-world data are presented to substantiate the theoretical claims. The mathematical analysis, algorithms, and software developed in the paper are equally suitable to other angular distributions of rays and therefore we bring the benefits of linograms to arbitrary radial patterns.
Nested Dithered Quantization for Communication Reduction in Distributed Training
Venue: arXiv cs.DC 2019
Year: 2019

In distributed training, the communication cost due to the transmission of gradients or the parameters of the deep model is a major bottleneck in scaling up the number of processing nodes. To address this issue, we propose \emph{dithered quantization} for the transmission of the stochastic gradients and show that training with \emph{Dithered Quantized Stochastic Gradients (DQSG)} is similar to the training with unquantized SGs perturbed by an independent bounded uniform noise, in contrast to the other quantization methods where the perturbation depends on the gradients and hence, complicating the convergence analysis. We study the convergence of training algorithms using DQSG and the trade off between the number of quantization levels and the training time. Next, we observe that there is a correlation among the SGs computed by workers that can be utilized to further reduce the communication overhead without any performance loss. Hence, we develop a simple yet effective quantization scheme, nested dithered quantized SG (NDQSG), that can reduce the communication significantly \emph{without requiring the workers communicating extra information to each other}. We prove that although NDQSG requires significantly less bits, it can achieve the same quantization variance bound as DQSG. Our simulation results confirm the effectiveness of training using DQSG and NDQSG in reducing the communication bits or the convergence time compared to the existing methods without sacrificing the accuracy of the trained model.
Peak Alignment of Gas Chromatography-Mass Spectrometry Data with Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

We present ChromAlignNet, a deep learning model for alignment of peaks in Gas Chromatography-Mass Spectrometry (GC-MS) data. In GC-MS data, a compound's retention time (RT) may not stay fixed across multiple chromatograms. To use GC-MS data for biomarker discovery requires alignment of identical analyte's RT from different samples. Current methods of alignment are all based on a set of formal, mathematical rules. We present a solution to GC-MS alignment using deep learning neural networks, which are more adept at complex, fuzzy data sets. We tested our model on several GC-MS data sets of various complexities and analysed the alignment results quantitatively. We show the model has very good performance (AUC $\sim 1$ for simple data sets and AUC $\sim 0.85$ for very complex data sets). Further, our model easily outperforms existing algorithms on complex data sets. Compared with existing methods, ChromAlignNet is very easy to use as it requires no user input of reference chromatograms and parameters. This method can easily be adapted to other similar data such as those from liquid chromatography. The source code is written in Python and available online.
Fence GAN: Towards Better Anomaly Detection
Venue: arXiv cs.LG 2019
Year: 2019

Anomaly detection is a classical problem where the aim is to detect anomalous data that do not belong to the normal data distribution. Current state-of-the-art methods for anomaly detection on complex high-dimensional data are based on the generative adversarial network (GAN). However, the traditional GAN loss is not directly aligned with the anomaly detection objective: it encourages the distribution of the generated samples to overlap with the real data and so the resulting discriminator has been found to be ineffective as an anomaly detector. In this paper, we propose simple modifications to the GAN loss such that the generated samples lie at the boundary of the real data distribution. With our modified GAN loss, our anomaly detection method, called Fence GAN (FGAN), directly uses the discriminator score as an anomaly threshold. Our experimental results using the MNIST, CIFAR10 and KDD99 datasets show that Fence GAN yields the best anomaly classification accuracy compared to state-of-the-art methods.
DSAL-GAN: Denoising based Saliency Prediction with Generative Adversarial Networks
Venue: arXiv cs.CV 2019
Year: 2019

Synthesizing high quality saliency maps from noisy images is a challenging problem in computer vision and has many practical applications. Samples generated by existing techniques for saliency detection cannot handle the noise perturbations smoothly and fail to delineate the salient objects present in the given scene. In this paper, we present a novel end-to-end coupled Denoising based Saliency Prediction with Generative Adversarial Network (DSAL-GAN) framework to address the problem of salient object detection in noisy images. DSAL-GAN consists of two generative adversarial-networks (GAN) trained end-to-end to perform denoising and saliency prediction altogether in a holistic manner. The first GAN consists of a generator which denoises the noisy input image, and in the discriminator counterpart we check whether the output is a denoised image or ground truth original image. The second GAN predicts the saliency maps from raw pixels of the input denoised image using a data-driven metric based on saliency prediction method with adversarial loss. Cycle consistency loss is also incorporated to further improve salient region prediction. We demonstrate with comprehensive evaluation that the proposed framework outperforms several baseline saliency models on various performance benchmarks.
Deep Learning for Face Recognition: Pride or Prejudiced?
Venue: arXiv cs.CV 2019
Year: 2019

Do very high accuracies of deep networks suggest pride of effective AI or are deep networks prejudiced? Do they suffer from in-group biases (own-race-bias and own-age-bias), and mimic the human behavior? Is in-group specific information being encoded sub-consciously by the deep networks? This research attempts to answer these questions and presents an in-depth analysis of `bias' in deep learning based face recognition systems. This is the first work which decodes if and where bias is encoded for face recognition. Taking cues from cognitive studies, we inspect if deep networks are also affected by social in- and out-group effect. Networks are analyzed for own-race and own-age bias, both of which have been well established in human beings. The sub-conscious behavior of face recognition models is examined to understand if they encode race or age specific features for face recognition. Analysis is performed based on 36 experiments conducted on multiple datasets. Four deep learning networks either trained from scratch or pre-trained on over 10M images are used. Variations across class activation maps and feature visualizations provide novel insights into the functioning of deep learning systems, suggesting behavior similar to humans. It is our belief that a better understanding of state-of-the-art deep learning networks would enable researchers to address the given challenge of bias in AI, and develop fairer systems.
How Much Can D2D Communication Reduce Content Delivery Latency in Fog Networks with Edge Caching?
Venue: arXiv cs.IT 2019
Year: 2019

A Fog-Radio Access Network (F-RAN) is studied in which cache-enabled Edge Nodes (ENs) with dedicated fronthaul connections to the cloud aim at delivering contents to mobile users. Using an information-theoretic approach, this work tackles the problem of quantifying the potential latency reduction that can be obtained by enabling Device-to-Device (D2D) communication over out-of-band broadcast links. Following prior work, the Normalized Delivery Time (NDT) --- a metric that captures the high signal-to-noise ratio worst-case latency --- is adopted as the performance criterion of interest. Joint edge caching, downlink transmission, and D2D communication policies based on compress-and-forward are proposed that are shown to be information-theoretically optimal to within a constant multiplicative factor of two for all values of the problem parameters, and to achieve the minimum NDT for a number of special cases. The analysis provides insights on the role of D2D cooperation in improving the delivery latency.
Experiments on Open-Set Speaker Identification with Discriminatively Trained Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

This paper presents a study on discriminative artificial neural network classifiers in the context of open-set speaker identification. Both 2-class and multi-class architectures are tested against the conventional Gaussian mixture model based classifier on enrolled speaker sets of different sizes. The performance evaluation shows that the multi-class neural network system has superior performance for large population sizes.
Multiparty Session Type-safe Web Development with Static Linearity
Venue: arXiv cs.PL 2019
Year: 2019

Modern web applications can now offer desktop-like experiences from within the browser, thanks to technologies such as WebSockets, which enable low-latency duplex communication between the browser and the server. While these advances are great for the user experience, they represent a new responsibility for web developers who now need to manage and verify the correctness of more complex and potentially stateful interactions in their application. In this paper, we present a technique for developing interactive web applications that are statically guaranteed to communicate following a given protocol. First, the global interaction protocol is described in the Scribble protocol language -- based on multiparty session types. Scribble protocols are checked for well-formedness, and then each role is projected to a Finite State Machine representing the structure of communication from the perspective of the role. We use source code generation and a novel type-level encoding of FSMs using multi-parameter type classes to leverage the type system of the target language and guarantee only programs that communicate following the protocol will type check. Our work targets PureScript -- a functional language that compiles to JavaScript -- which crucially has an expressive enough type system to provide static linearity guarantees. We demonstrate the effectiveness of our approach through a web-based Battleship game where communication is performed through WebSocket connections.
Temporal and Aspectual Entailment
Venue: arXiv cs.CL 2019
Year: 2019

Inferences regarding "Jane's arrival in London" from predications such as "Jane is going to London" or "Jane has gone to London" depend on tense and aspect of the predications. Tense determines the temporal location of the predication in the past, present or future of the time of utterance. The aspectual auxiliaries on the other hand specify the internal constituency of the event, i.e. whether the event of "going to London" is completed and whether its consequences hold at that time or not. While tense and aspect are among the most important factors for determining natural language inference, there has been very little work to show whether modern NLP models capture these semantic concepts. In this paper we propose a novel entailment dataset and analyse the ability of a range of recently proposed NLP models to perform inference on temporal predications. We show that the models encode a substantial amount of morphosyntactic information relating to tense and aspect, but fail to model inferences that require reasoning with these semantic properties.
Verification and Validation of Semantic Annotations
Venue: arXiv cs.IR 2019
Year: 2019

In this paper, we propose a framework to perform verification and validation of semantically annotated data. The annotations, extracted from websites, are verified against the schema.org vocabulary and Domain Specifications to ensure the syntactic correctness and completeness of the annotations. The Domain Specifications allow checking the compliance of annotations against corresponding domain-specific constraints. The validation mechanism will detect errors and inconsistencies between the content of the analyzed schema.org annotations and the content of the web pages where the annotations were found.
On Geometric Structure of Activation Spaces in Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we investigate the geometric structure of activation spaces of fully connected layers in neural networks and then show applications of this study. We propose an efficient approximation algorithm to characterize the convex hull of massive points in high dimensional space. Based on this new algorithm, four common geometric properties shared by the activation spaces are concluded, which gives a rather clear description of the activation spaces. We then propose an alternative classification method grounding on the geometric structure description, which works better than neural networks alone. Surprisingly, this data classification method can be an indicator of overfitting in neural networks. We believe our work reveals several critical intrinsic properties of modern neural networks and further gives a new metric for evaluating them.
Cooperative Embeddings for Instance, Attribute and Category Retrieval
Venue: arXiv cs.CV 2019
Year: 2019

The goal of this paper is to retrieve an image based on instance, attribute and category similarity notions. Different from existing works, which usually address only one of these entities in isolation, we introduce a cooperative embedding to integrate them while preserving their specific level of semantic representation. An algebraic structure defines a superspace filled with instances. Attributes are axis-aligned to form subspaces, while categories influence the arrangement of similar instances. These relationships enable them to cooperate for their mutual benefits for image retrieval. We derive a proxy-based softmax embedding loss to learn simultaneously all similarity measures in both superspace and subspaces. We evaluate our model on datasets from two different domains. Experiments on image retrieval tasks show the benefits of the cooperative embeddings for modeling multiple image similarities, and for discovering style evolution of instances between- and within-categories.
Deep Learning in steganography and steganalysis from 2015 to 2018
Venue: arXiv cs.CR 2019
Year: 2019

For almost 10 years, the detection of a hidden message in an image has been mainly carried out by the computation of Rich Models (RM), followed by classification using an Ensemble Classifier (EC). In 2015, the first study using a convolutional neural network (CNN) obtained the first results of steganalysis by Deep Learning approaching the performances of the two-step approach (EC + RM). Between 2015-2018, numerous publications have shown that it is possible to obtain improved performances, notably in spatial steganalysis, JPEG steganalysis, Selection-Channel-Aware steganalysis, and in quantitative steganalysis. This chapter deals with deep learning in steganalysis from the point of view of current methods, by presenting different neural networks from the period 2015-2018, that have been evaluated with a methodology specific to the discipline of steganalysis. The chapter is not intended to repeat the basic concepts of machine learning or deep learning. So, we will present the structure of a deep neural network, in a generic way and present the networks proposed in existing literature for the different scenarios of steganalysis, and finally, we will discuss steganography by deep learning.
Secure Communications in a Unified Non-Orthogonal Multiple Access Framework
Venue: arXiv cs.IT 2019
Year: 2019

This paper investigates the impact of physical layer secrecy on the performance of a unified non-orthogonal multiple access (NOMA) framework, where both external and internal eavesdropping scenarios are examined. The spatial locations of legitimate users (LUs) and eavesdroppers are modeled by invoking stochastic geometry. To characterize the security performance, new exact and asymptotic expressions of secrecy outage probability (SOP) are derived for both code-domain NOMA (CD-NOMA) and power-domain NOMA (PD-NOMA), in which imperfect successive interference cancellation (ipSIC) and perfect SIC (pSIC) are taken into account. For the external eavesdropping scenario, the secrecy diversity orders by a pair of LUs (the n-th user and m-th user) for CD/PD-NOMA are obtained. Analytical results make known that the diversity orders of the $n$-th user with ipSIC/pSIC for CD-NOMA and PD-NOMA are equal to zero/K and zero/one, respectively. The diversity orders of the m-th user are equal to K/one for CD/PD-NOMA. For the internal eavesdropping scenario, we examine the analysis of secrecy diversity order and observe that the m-th user to wiretap the n-th user with ipSIC/pSIC for CD-NOMA and PD-NOMA provide the diversity orders of zero/K and zero/one, respectively, which is consistent with external eavesdropping scenario. Numerical results are present to confirm the accuracy of the analytical results developed and show that: i) The secrecy outage behavior of the $n$-th user is superior to that of the m-th user; ii) By increasing the number of subcarriers, CD-NOMA is capable of achieving a larger secrecy diversity gain compared to PD-NOMA.
Finite Strain Homogenization Using a Reduced Basis and Efficient Sampling
Venue: arXiv cs.CE 2019
Year: 2019

The computational homogenization of hyperelastic solids in the geometrically nonlinear context has yet to be treated with sufficient efficiency in order to allow for real-world applications in true multiscale settings. This problem is addressed by a problem-specific surrogate model founded on a reduced basis approximation of the deformation gradient on the microscale. The setup phase is based upon a snapshot POD on deformation gradient fluctuations, in contrast to the widespread displacement-based approach. In order to reduce the computational offline costs, the space of relevant macroscopic stretch tensors is sampled efficiently by employing the Hencky strain. Numerical results show speed-up factors in the order of 5-100 and significantly improved robustness while retaining good accuracy. An open-source demonstrator tool with 50 lines of code emphasizes the simplicity and efficiency of the method.