OpenRaiser/Intern-Atlas huggingface dataset:
A Cost-efficient IoT Forensics Framework with Blockchain
Venue: arXiv cs.CR 2020
Year: 2020

IoT devices have been adopted widely in the last decade which enabled collection of various data from different environments. The collected data is crucial in certain applications where IoT devices generate data for critical infrastructure or systems whose failure may result in catastrophic results. Specifically, for such critical applications, data storage poses challenges since the data may be compromised during the storage and the integrity might be violated without being noticed. In such cases, integrity and data provenance are required in order to be able to detect the source of any incident and prove it in legal cases if there is a dispute with the involved parties. To address these issues, blockchain provides excellent opportunities since it can protect the integrity of the data thanks to its distributed structure. However, it comes with certain costs as storing huge amount of data in a public blockchain will come with significant transaction fees. In this paper, we propose a highly cost effective and reliable digital forensics framework by exploiting multiple inexpensive blockchain networks as a temporary storage before the data is committed to Ethereum. To reduce Ethereum costs,we utilize Merkle trees which hierarchically stores hashes of the collected event data from IoT devices. We evaluated the approach on popular blockchains such as EOS, Stellar, and Ethereum by presenting a cost and security analysis. The results indicate that we can achieve significant cost savings without compromising the integrity of the data.
A Novel Perspective to Zero-shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion
Venue: arXiv cs.CV 2020
Year: 2020

Zero-shot learning aims at recognizing unseen classes (no training example) with knowledge transferred from seen classes. This is typically achieved by exploiting a semantic feature space shared by both seen and unseen classes, i.e., attribute or word vector, as the bridge. One common practice in zero-shot learning is to train a projection between the visual and semantic feature spaces with labeled seen classes examples. When inferring, this learned projection is applied to unseen classes and recognizes the class labels by some metrics. However, the visual and semantic feature spaces are mutually independent and have quite different manifold structures. Under such a paradigm, most existing methods easily suffer from the domain shift problem and weaken the performance of zero-shot recognition. To address this issue, we propose a novel model called AMS-SFE. It considers the alignment of manifold structures by semantic feature expansion. Specifically, we build upon an autoencoder-based model to expand the semantic features from the visual inputs. Additionally, the expansion is jointly guided by an embedded manifold extracted from the visual feature space of the data. Our model is the first attempt to align both feature spaces by expanding semantic features and derives two benefits: first, we expand some auxiliary features that enhance the semantic feature space; second and more importantly, we implicitly align the manifold structures between the visual and semantic feature spaces; thus, the projection can be better trained and mitigate the domain shift problem. Extensive experiments show significant performance improvement, which verifies the effectiveness of our model.
Knowledge Graph Embeddings and Explainable AI
Venue: arXiv cs.AI 2020
Year: 2020

Knowledge graph embeddings are now a widely adopted approach to knowledge representation in which entities and relationships are embedded in vector spaces. In this chapter, we introduce the reader to the concept of knowledge graph embeddings by explaining what they are, how they can be generated and how they can be evaluated. We summarize the state-of-the-art in this field by describing the approaches that have been introduced to represent knowledge in the vector space. In relation to knowledge representation, we consider the problem of explainability, and discuss models and methods for explaining predictions obtained via knowledge graph embeddings.
MuSe 2020 -- The First International Multimodal Sentiment Analysis in Real-life Media Challenge and Workshop
Venue: arXiv cs.MM 2020
Year: 2020

Multimodal Sentiment Analysis in Real-life Media (MuSe) 2020 is a Challenge-based Workshop focusing on the tasks of sentiment recognition, as well as emotion-target engagement and trustworthiness detection by means of more comprehensively integrating the audio-visual and language modalities. The purpose of MuSe 2020 is to bring together communities from different disciplines; mainly, the audio-visual emotion recognition community (signal-based), and the sentiment analysis community (symbol-based). We present three distinct sub-challenges: MuSe-Wild, which focuses on continuous emotion (arousal and valence) prediction; MuSe-Topic, in which participants recognise domain-specific topics as the target of 3-class (low, medium, high) emotions; and MuSe-Trust, in which the novel aspect of trustworthiness is to be predicted. In this paper, we provide detailed information on MuSe-CaR, the first of its kind in-the-wild database, which is utilised for the challenge, as well as the state-of-the-art features and modelling approaches applied. For each sub-challenge, a competitive baseline for participants is set; namely, on test we report for MuSe-Wild a combined (valence and arousal) CCC of .2568, for MuSe-Topic a score (computed as 0.34$\cdot$ UAR + 0.66$\cdot$F1) of 76.78 % on the 10-class topic and 40.64 % on the 3-class emotion prediction, and for MuSe-Trust a CCC of .4359.
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding
Venue: arXiv cs.CL 2020
Year: 2020

Inflectional variation is a common feature of World Englishes such as Colloquial Singapore English and African American Vernacular English. Although comprehension by human readers is usually unimpaired by non-standard inflections, current NLP systems are not yet robust. We propose Base-Inflection Encoding (BITE), a method to tokenize English text by reducing inflected words to their base forms before reinjecting the grammatical information as special symbols. Fine-tuning pretrained NLP models for downstream tasks using our encoding defends against inflectional adversaries while maintaining performance on clean data. Models using BITE generalize better to dialects with non-standard inflections without explicit training and translation models converge faster when trained with BITE. Finally, we show that our encoding improves the vocabulary efficiency of popular data-driven subword tokenizers. Since there has been no prior work on quantitatively evaluating vocabulary efficiency, we propose metrics to do so.
Parallel processor scheduling: formulation as multi-objective linguistic optimization and solution using Perceptual Reasoning based methodology
Venue: arXiv cs.AI 2020
Year: 2020

In the era of Industry 4.0, the focus is on the minimization of human element and maximizing the automation in almost all the industrial and manufacturing establishments. These establishments contain numerous processing systems, which can execute a number of tasks, in parallel with minimum number of human beings. This parallel execution of tasks is done in accordance to a scheduling policy. However, the minimization of human element beyond a certain point is difficult. In fact, the expertise and experience of a group of humans, called the experts, becomes imminent to design a fruitful scheduling policy. The aim of the scheduling policy is to achieve the optimal value of an objective, like production time, cost, etc. In real-life situations, there are more often than not, multiple objectives in any parallel processing scenario. Furthermore, the experts generally provide their opinions, about various scheduling criteria (pertaining to the scheduling policies) in linguistic terms or words. Word semantics are best modeled using fuzzy sets (FSs). Thus, all these factors have motivated us to model the parallel processing scenario as a multi-objective linguistic optimization problem (MOLOP) and use the novel perceptual reasoning (PR) based methodology for solving it. We have also compared the results of the PR based solution methodology with those obtained from the 2-tuple based solution methodology. PR based solution methodology offers three main advantages viz., it generates unique recommendations, here the linguistic recommendations match a codebook word, and also the word model comes before the word. 2-tuple based solution methodology fails to give all these advantages. Thus, we feel that our work is novel and will provide directions for the future research.
Natural Language Premise Selection: Finding Supporting Statements for Mathematical Text
Venue: arXiv cs.CL 2020
Year: 2020

Mathematical text is written using a combination of words and mathematical expressions. This combination, along with a specific way of structuring sentences makes it challenging for state-of-art NLP tools to understand and reason on top of mathematical discourse. In this work, we propose a new NLP task, the natural premise selection, which is used to retrieve supporting definitions and supporting propositions that are useful for generating an informal mathematical proof for a particular statement. We also make available a dataset, NL-PS, which can be used to evaluate different approaches for the natural premise selection task. Using different baselines, we demonstrate the underlying interpretation challenges associated with the task.
Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation
Venue: arXiv cs.CL 2020
Year: 2020

We introduce CGA, a conditional VAE architecture, to control, generate, and augment text. CGA is able to generate natural English sentences controlling multiple semantic and syntactic attributes by combining adversarial learning with a context-aware loss and a cyclical word dropout routine. We demonstrate the value of the individual model components in an ablation study. The scalability of our approach is ensured through a single discriminator, independently of the number of attributes. We show high quality, diversity and attribute control in the generated sentences through a series of automatic and human assessments. As the main application of our work, we test the potential of this new NLG model in a data augmentation scenario. In a downstream NLP task, the sentences generated by our CGA model show significant improvements over a strong baseline, and a classification performance often comparable to adding same amount of additional real data.
Compound Secure Groupcast: Key Assignment for Selected Broadcasting
Venue: arXiv cs.IT 2020
Year: 2020

The compound secure groupcast problem is considered, where the key variables at $K$ receivers are designed so that a transmitter can securely groupcast a message to any $N$ out of the $K$ receivers through a noiseless broadcast channel. The metric is the information theoretic tradeoff between key storage $α$, i.e., the number of bits of the key variable per message bit, and broadcast bandwidth $β$, i.e., the number of bits of the broadcast information per message bit. We have three main results. First, when broadcast bandwidth is minimized, i.e., when $β= 1$, we show that the minimum key storage is $α= N$. Second, when key storage is minimized, i.e., when $α= 1$, we show that broadcast bandwidth $β= \min(N, K-N+1)$ is achievable and is optimal (minimum) if $N=2$ or $K-1$. Third, when $N=2$, the optimal key storage and broadcast bandwidth tradeoff is characterized as $α+β\geq 3, α\geq 1, β\geq 1$.
Consistent Video Depth Estimation
Venue: arXiv cs.CV 2020
Year: 2020

We present an algorithm for reconstructing dense, geometrically consistent depth for all pixels in a monocular video. We leverage a conventional structure-from-motion reconstruction to establish geometric constraints on pixels in the video. Unlike the ad-hoc priors in classical reconstruction, we use a learning-based prior, i.e., a convolutional neural network trained for single-image depth estimation. At test time, we fine-tune this network to satisfy the geometric constraints of a particular input video, while retaining its ability to synthesize plausible depth details in parts of the video that are less constrained. We show through quantitative validation that our method achieves higher accuracy and a higher degree of geometric consistency than previous monocular reconstruction methods. Visually, our results appear more stable. Our algorithm is able to handle challenging hand-held captured input videos with a moderate degree of dynamic motion. The improved quality of the reconstruction enables several applications, such as scene reconstruction and advanced video-based visual effects.
Method for Customizable Automated Tagging: Addressing the Problem of Over-tagging and Under-tagging Text Documents
Venue: arXiv cs.IR 2020
Year: 2020

Using author provided tags to predict tags for a new document often results in the overgeneration of tags. In the case where the author doesn't provide any tags, our documents face the severe under-tagging issue. In this paper, we present a method to generate a universal set of tags that can be applied widely to a large document corpus. Using IBM Watson's NLU service, first, we collect keywords/phrases that we call "complex document tags" from 8,854 popular reports in the corpus. We apply LDA model over these complex document tags to generate a set of 765 unique "simple tags". In applying the tags to a corpus of documents, we run each document through the IBM Watson NLU and apply appropriate simple tags. Using only 765 simple tags, our method allows us to tag 87,397 out of 88,583 total documents in the corpus with at least one tag. About 92.1% of the total 87,397 documents are also determined to be sufficiently-tagged. In the end, we discuss the performance of our method and its limitations.
Attribution Analysis of Grammatical Dependencies in LSTMs
Venue: arXiv cs.CL 2020
Year: 2020

LSTM language models have been shown to capture syntax-sensitive grammatical dependencies such as subject-verb agreement with a high degree of accuracy (Linzen et al., 2016, inter alia). However, questions remain regarding whether they do so using spurious correlations, or whether they are truly able to match verbs with their subjects. This paper argues for the latter hypothesis. Using layer-wise relevance propagation (Bach et al., 2015), a technique that quantifies the contributions of input features to model behavior, we show that LSTM performance on number agreement is directly correlated with the model's ability to distinguish subjects from other nouns. Our results suggest that LSTM language models are able to infer robust representations of syntactic dependencies.
Occlusion resistant learning of intuitive physics from videos
Venue: arXiv cs.CV 2020
Year: 2020

To reach human performance on complex tasks, a key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences. Yet, most of these methods are restricted to the case where no, or only limited, occlusions occur. In this work we propose a probabilistic formulation of learning intuitive physics in 3D scenes with significant inter-object occlusions. In our formulation, object positions are modeled as latent variables enabling the reconstruction of the scene. We then propose a series of approximations that make this problem tractable. Object proposals are linked across frames using a combination of a recurrent interaction network, modeling the physics in object space, and a compositional renderer, modeling the way in which objects project onto pixel space. We demonstrate significant improvements over state-of-the-art in the intuitive physics benchmark of IntPhys. We apply our method to a second dataset with increasing levels of occlusions, showing it realistically predicts segmentation masks up to 30 frames in the future. Finally, we also show results on predicting motion of objects in real videos.
Linguistic Typology Features from Text: Inferring the Sparse Features of World Atlas of Language Structures
Venue: arXiv cs.CL 2020
Year: 2020

The use of linguistic typological resources in natural language processing has been steadily gaining more popularity. It has been observed that the use of typological information, often combined with distributed language representations, leads to significantly more powerful models. While linguistic typology representations from various resources have mostly been used for conditioning the models, there has been relatively little attention on predicting features from these resources from the input data. In this paper we investigate whether the various linguistic features from World Atlas of Language Structures (WALS) can be reliably inferred from multi-lingual text. Such a predictor can be used to infer structural features for a language never observed in training data. We frame this task as a multi-label classification involving predicting the set of non-mutually exclusive and extremely sparse multi-valued labels (WALS features). We construct a recurrent neural network predictor based on byte embeddings and convolutional layers and test its performance on 556 languages, providing analysis for various linguistic types, macro-areas, language families and individual features. We show that some features from various linguistic types can be predicted reliably.
PANDA: Policy-aware Location Privacy for Epidemic Surveillance
Venue: arXiv cs.DB 2020
Year: 2020

In this demonstration, we present a privacy-preserving epidemic surveillance system. Recently, many countries that suffer from coronavirus crises attempt to access citizen's location data to eliminate the outbreak. However, it raises privacy concerns and may open the doors to more invasive forms of surveillance in the name of public health. It also brings a challenge for privacy protection techniques: how can we leverage people's mobile data to help combat the pandemic without scarifying our location privacy. We demonstrate that we can have the best of the two worlds by implementing policy-based location privacy for epidemic surveillance. Specifically, we formalize the privacy policy using graphs in light of differential privacy, called policy graph. Our system has three primary functions for epidemic surveillance: location monitoring, epidemic analysis, and contact tracing. We provide an interactive tool allowing the attendees to explore and examine the usability of our system: (1) the utility of location monitor and disease transmission model estimation, (2) the procedure of contact tracing in our systems, and (3) the privacy-utility trade-offs w.r.t. different policy graphs. The attendees can find that it is possible to have the full functionality of epidemic surveillance while preserving location privacy.
Generating Randomness from a Computable, Non-random Sequence of Qubits
Venue: arXiv cs.IT 2020
Year: 2020

Nies and Scholz introduced the notion of a state to describe an infinite sequence of qubits and defined quantum-Martin-Lof randomness for states, analogously to the well known concept of Martin-Löf randomness for elements of Cantor space (the space of infinite sequences of bits). We formalize how 'measurement' of a state in a basis induces a probability measure on Cantor space. A state is 'measurement random' (mR) if the measure induced by it, under any computable basis, assigns probability one to the set of Martin-Löf randoms. Equivalently, a state is mR if and only if measuring it in any computable basis yields a Martin-Löf random with probability one. While quantum-Martin-Löf random states are mR, the converse fails: there is a mR state, x which is not quantum-Martin-Löf random. In fact, something stronger is true. While x is computable and can be easily constructed, measuring it in any computable basis yields an arithmetically random sequence with probability one. I.e., classical arithmetic randomness can be generated from a computable, non-quantum random sequence of qubits.
Why FHilb is Not an Interesting (Co)Differential Category
Venue: arXiv cs.LO 2020
Year: 2020

Differential categories provide an axiomatization of the basics of differentiation and categorical models of differential linear logic. As differentiation is an important tool throughout quantum mechanics and quantum information, it makes sense to study applications of the theory of differential categories to categorical quantum foundations. In categorical quantum foundations, compact closed categories (and therefore traced symmetric monoidal categories) are one of the main objects of study, in particular the category of finite-dimensional Hilbert spaces FHilb. In this paper, we will explain why the only differential category structure on FHilb is the trivial one. This follows from a sort of in-compatibility between the trace of FHilb and possible differential category structure. That said, there are interesting non-trivial examples of traced/compact closed differential categories, which we also discuss. The goal of this paper is to introduce differential categories to the broader categorical quantum foundation community and hopefully open the door to further work in combining these two fields. While the main result of this paper may seem somewhat "negative" in achieving this goal, we discuss interesting potential applications of differential categories to categorical quantum foundations.
Scott Continuity in Generalized Probabilistic Theories
Venue: arXiv cs.LO 2020
Year: 2020

Scott continuity is a concept from domain theory that had an unexpected previous life in the theory of von Neumann algebras. Scott-continuous states are known as normal states, and normal states are exactly the states coming from density matrices. Given this, and the usefulness of Scott continuity in domain theory, it is natural to ask whether this carries over to generalized probabilistic theories. We show that the answer is no - there are infinite-dimensional convex sets for which the set of Scott-continuous states on the corresponding set of 2-valued POVMs does not recover the original convex set, but is strictly larger. This shows the necessity of the use of topologies for state-effect duality in the general case, rather than purely order theoretic notions.
Facilitating Access to Multilingual COVID-19 Information via Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

Every day, more people are becoming infected and dying from exposure to COVID-19. Some countries in Europe like Spain, France, the UK and Italy have suffered particularly badly from the virus. Others such as Germany appear to have coped extremely well. Both health professionals and the general public are keen to receive up-to-date information on the effects of the virus, as well as treatments that have proven to be effective. In cases where language is a barrier to access of pertinent information, machine translation (MT) may help people assimilate information published in different languages. Our MT systems trained on COVID-19 data are freely available for anyone to use to help translate information published in German, French, Italian, Spanish into English, as well as the reverse direction.