OpenRaiser/Intern-Atlas huggingface dataset:
GraphVICRegHSIC: Towards improved self-supervised representation learning for graphs with a hyrbid loss function
Venue: arXiv cs.LG 2021
Year: 2021

Self-supervised learning and pre-training strategieshave developed over the last few years especiallyfor Convolutional Neural Networks (CNNs). Re-cently application of such methods can also be no-ticed for Graph Neural Networks (GNNs) . In thispaper, we have used a graph based self-supervisedlearning strategy with different loss functions (Bar-low Twins[Zbontaret al., 2021], HSIC[Tsaiet al.,2021], VICReg[Bardeset al., 2021]) which haveshown promising results when applied with CNNspreviously. We have also proposed a hybrid lossfunction combining the advantages of VICReg andHSIC and called it as VICRegHSIC. The perfor-mance of these aforementioned methods have beencompared when applied to 7 different datasets suchas MUTAG, PROTEINS, IMDB-Binary, etc. Ex-periments showed that our hybrid loss function per-formed better than the remaining ones in 4 out of7 cases. Moreover, the impact of different batchsizes, projector dimensions and data augmentationstrategies have also been explored.
Word Embedding Transformation for Robust Unsupervised Bilingual Lexicon Induction
Venue: arXiv cs.CL 2021
Year: 2021

Great progress has been made in unsupervised bilingual lexicon induction (UBLI) by aligning the source and target word embeddings independently trained on monolingual corpora. The common assumption of most UBLI models is that the embedding spaces of two languages are approximately isomorphic. Therefore the performance is bound by the degree of isomorphism, especially on etymologically and typologically distant languages. To address this problem, we propose a transformation-based method to increase the isomorphism. Embeddings of two languages are made to match with each other by rotating and scaling. The method does not require any form of supervision and can be applied to any language pair. On a benchmark data set of bilingual lexicon induction, our approach can achieve competitive or superior performance compared to state-of-the-art methods, with particularly strong results being found on distant languages.
Design Your Life: User-Initiated Design of Technology to Support Independent Living of Young Autistic Adults
Venue: arXiv cs.HC 2021
Year: 2021

This paper describes the development of and first experiences with 'Design Your Life': a novel method aimed at user-initiated design of technologies supporting young autistic adults in independent living. A conceptual, phenomenological background resulting in four core principles is described. Taking a practice-oriented Research-through-Design approach, three co-design case studies were conducted, in which promising methods from the co-design literature with the lived experiences and practical contexts of autistic young adults and their caregivers is contrasted. This explorative inquiry provided some first insights into several design directions of the Design Your Life-process. In a series of new case studies that shall follow, the Design Your Life-method will be iteratively developed, refined and ultimately validated in practice.
Towards an IMU-based Pen Online Handwriting Recognizer
Venue: arXiv cs.LG 2021
Year: 2021

Most online handwriting recognition systems require the use of specific writing surfaces to extract positional data. In this paper we present a online handwriting recognition system for word recognition which is based on inertial measurement units (IMUs) for digitizing text written on paper. This is obtained by means of a sensor-equipped pen that provides acceleration, angular velocity, and magnetic forces streamed via Bluetooth. Our model combines convolutional and bidirectional LSTM networks, and is trained with the Connectionist Temporal Classification loss that allows the interpretation of raw sensor data into words without the need of sequence segmentation. We use a dataset of words collected using multiple sensor-enhanced pens and evaluate our model on distinct test sets of seen and unseen words achieving a character error rate of 17.97% and 17.08%, respectively, without the use of a dictionary or language model
Pattern Detection in the Activation Space for Identifying Synthesized Content
Venue: arXiv cs.CV 2021
Year: 2021

Generative Adversarial Networks (GANs) have recently achieved unprecedented success in photo-realistic image synthesis from low-dimensional random noise. The ability to synthesize high-quality content at a large scale brings potential risks as the generated samples may lead to misinformation that can create severe social, political, health, and business hazards. We propose SubsetGAN to identify generated content by detecting a subset of anomalous node-activations in the inner layers of pre-trained neural networks. These nodes, as a group, maximize a non-parametric measure of divergence away from the expected distribution of activations created from real data. This enable us to identify synthesised images without prior knowledge of their distribution. SubsetGAN efficiently scores subsets of nodes and returns the group of nodes within the pre-trained classifier that contributed to the maximum score. The classifier can be a general fake classifier trained over samples from multiple sources or the discriminator network from different GANs. Our approach shows consistently higher detection power than existing detection methods across several state-of-the-art GANs (PGGAN, StarGAN, and CycleGAN) and over different proportions of generated content.
Dimensioning of V2X Services in 5G Networks through Forecast-based Scaling
Venue: arXiv cs.NI 2021
Year: 2021

With the increasing adoption of intelligent transportation systems and the upcoming era of autonomous vehicles, vehicular services (such as, remote driving, cooperative awareness, and hazard warning) will face an ever changing and dynamic environment. Traffic flows on the roads is a critical condition for these services and, therefore, it is of paramount importance to forecast how they will evolve over time. By knowing future events (such as, traffic jams), vehicular services can be dimensioned in an on-demand fashion in order to minimize Service Level Agreements (SLAs) violations, thus reducing the chances of car accidents. This research departs from an evaluation of traditional time-series techniques with recent Machine Learning (ML)-based solutions to forecast traffic flows in the roads of Torino (Italy). Given the accuracy of the selected forecasting techniques, a forecast-based scaling algorithm is proposed and evaluated over a set of dimensioning experiments of three distinct vehicular services with strict latency requirements. Results show that the proposed scaling algorithm enables resource savings of up to a 5% at the cost of incurring in an increase of less than 0.4% of latency violations.
Unsupervised Video Summarization via Multi-source Features
Venue: arXiv cs.CV 2021
Year: 2021

Video summarization aims at generating a compact yet representative visual summary that conveys the essence of the original video. The advantage of unsupervised approaches is that they do not require human annotations to learn the summarization capability and generalize to a wider range of domains. Previous work relies on the same type of deep features, typically based on a model pre-trained on ImageNet data. Therefore, we propose the incorporation of multiple feature sources with chunk and stride fusion to provide more information about the visual content. For a comprehensive evaluation on the two benchmarks TVSum and SumMe, we compare our method with four state-of-the-art approaches. Two of these approaches were implemented by ourselves to reproduce the reported results. Our evaluation shows that we obtain state-of-the-art results on both datasets, while also highlighting the shortcomings of previous work with regard to the evaluation methodology. Finally, we perform error analysis on videos for the two benchmark datasets to summarize and spot the factors that lead to misclassifications.
Securing Serverless Computing: Challenges, Solutions, and Opportunities
Venue: arXiv cs.CR 2021
Year: 2021

Serverless computing is a new cloud service model that reduces both cloud providers' and consumers' costs through extremely agile development, operation, and charging mechanisms and has been widely applied since its emergence. Nevertheless, some characteristics of serverless computing, such as fragmented application boundaries, have raised new security challenges. Considerable literature work has been committed to addressing these challenges. Commercial and open-source serverless platforms implement many security measures to enhance serverless environments. This paper presents the first survey of serverless security that considers both literature work and industrial security measures. We summarize the primary security challenges, analyze corresponding solutions from the literature and industry, and identify potential research opportunities. Then, we conduct a gap analysis of the academic and industrial solutions as well as commercial and open-source serverless platforms' security capabilities, and finally, we present a complete picture of current serverless security research.
A Comprehensive Survey on Community Detection with Deep Learning
Venue: arXiv cs.SI 2021
Year: 2021

A community reveals the features and connections of its members that are different from those in other communities in a network. Detecting communities is of great significance in network analysis. Despite the classical spectral clustering and statistical inference methods, we notice a significant development of deep learning techniques for community detection in recent years with their advantages in handling high dimensional network data. Hence, a comprehensive overview of community detection's latest progress through deep learning is timely to academics and practitioners. This survey devises and proposes a new taxonomy covering different state-of-the-art methods, including deep learning-based models upon deep neural networks, deep nonnegative matrix factorization and deep sparse filtering. The main category, i.e., deep neural networks, is further divided into convolutional networks, graph attention networks, generative adversarial networks and autoencoders. The survey also summarizes the popular benchmark data sets, evaluation metrics, and open-source implementations to address experimentation settings. We then discuss the practical applications of community detection in various domains and point to implementation scenarios. Finally, we outline future directions by suggesting challenging topics in this fast-growing deep learning field.
Automatic Construction of Sememe Knowledge Bases via Dictionaries
Venue: arXiv cs.CL 2021
Year: 2021

A sememe is defined as the minimum semantic unit in linguistics. Sememe knowledge bases (SKBs), which comprise words annotated with sememes, enable sememes to be applied to natural language processing. So far a large body of research has showcased the unique advantages and effectiveness of SKBs in various tasks. However, most languages have no SKBs, and manual construction of SKBs is time-consuming and labor-intensive. To tackle this challenge, we propose a simple and fully automatic method of building an SKB via an existing dictionary. We use this method to build an English SKB and a French SKB, and conduct comprehensive evaluations from both intrinsic and extrinsic perspectives. Experimental results demonstrate that the automatically built English SKB is even superior to HowNet, the most widely used SKB that takes decades to build manually. And both the English and French SKBs can bring obvious performance enhancement in multiple downstream tasks. All the code and data of this paper (except the copyrighted dictionaries) can be obtained at https://github.com/thunlp/DictSKB.
What makes you popular: beauty, personality or intelligence?
Venue: arXiv cs.SI 2021
Year: 2021

This study explores the determinants of popularity within friendship and advice networks. We involved almost 200 college students in an experiment to predict how personality traits, self-monitoring, creativity, intelligence, energy, and beauty influence the development of friendship and advice networks. Our results indicate that physical attractiveness is a key to develop both friendship and task-related interactions, whereas perceived intelligence and creativity play an important role in the advice network. Our findings seem to support the idea that there might be a kernel of truth in the stereotype that attractiveness correlates with positive social traits and successful outcomes.
Conceptual Schema Optimisation -- Database Optimisation before sliding down the Waterfall
Venue: arXiv cs.DB 2021
Year: 2021

In this article we discuss an approach to database optimisation in which a conceptual schema is optimised by applying a sequence of transformations. By performing these optimisations on the conceptual schema, a large part of the database optimisation can be done before actually sliding down the software development waterfall. When optimising schemas, one would like to preserve some level of equivalence between the schemas before and after a transformation. We distinguish between two classes of equivalence, one based on the mathematical semantics of the conceptual schemas, and one on conceptual preference by humans. As a medium for the schema transformations we use the universe of all (correct) conceptual schemas. A schema transformation process can then be seen as a journey (a schema-time worm) within this universe. The underlying theory is conveyed intuitively with sample transformations, and formalised within the framework of Object-Role Modelling. A metalanguage is introduced for the specification of transformations, and more importantly their semantics. While the discussion focusses on the data perspective, the approach has a high level of generality and is extensible to process and behaviour perspectives.
Business Suitability Principles for Workflow Modelling
Venue: arXiv cs.OH 2021
Year: 2021

By incorporating aspects of coordination and collaboration, workflow implementations of information systems require a sound conceptualisation of \EM{business processing} semantics. Traditionally, the success of conceptual modelling techniques has depended largely on the adequacy of conceptualisation, expressive power, comprehensibility and formal foundation. An equally important requirement, particularly with the increased conceptualisation of business aspects, is \EM{business suitability}. In this paper, the focus is on the business suitability of workflow modelling for a commonly encountered class of (operational) business processing, e.g. those of insurance claims, bank loans and land conveyancing. A general assessment is first conducted on some \EM{integrated} techniques characterising well-known paradigms - structured process modelling, object-oriented modelling, behavioural process modelling and business-oriented modelling. Through this, an insight into business suitability within the broader perspective of technique adequacy, is gained. A specific business suitability diagnosis then follows using a particular characterisation of business processing, i.e.\ one where the intuitive semantics and inter-relationship of business services and business processes are nuanced. As a result, five business suitability principles are elicited. These are proposed for a more detailed understanding and (synthetic) development of workflow modelling techniques. Accordingly, further insight into workflow specification languages and workflow globalisation in open distributed architectures may also be gained.
Dynamic Probabilistic Pruning: A general framework for hardware-constrained pruning at different granularities
Venue: arXiv cs.LG 2021
Year: 2021

Unstructured neural network pruning algorithms have achieved impressive compression rates. However, the resulting - typically irregular - sparse matrices hamper efficient hardware implementations, leading to additional memory usage and complex control logic that diminishes the benefits of unstructured pruning. This has spurred structured coarse-grained pruning solutions that prune entire filters or even layers, enabling efficient implementation at the expense of reduced flexibility. Here we propose a flexible new pruning mechanism that facilitates pruning at different granularities (weights, kernels, filters/feature maps), while retaining efficient memory organization (e.g. pruning exactly k-out-of-n weights for every output neuron, or pruning exactly k-out-of-n kernels for every feature map). We refer to this algorithm as Dynamic Probabilistic Pruning (DPP). DPP leverages the Gumbel-softmax relaxation for differentiable k-out-of-n sampling, facilitating end-to-end optimization. We show that DPP achieves competitive compression rates and classification accuracy when pruning common deep learning models trained on different benchmark datasets for image classification. Relevantly, the non-magnitude-based nature of DPP allows for joint optimization of pruning and weight quantization in order to even further compress the network, which we show as well. Finally, we propose novel information theoretic metrics that show the confidence and pruning diversity of pruning masks within a layer.
TexRel: a Green Family of Datasets for Emergent Communications on Relations
Venue: arXiv cs.CL 2021
Year: 2021

We propose a new dataset TexRel as a playground for the study of emergent communications, in particular for relations. By comparison with other relations datasets, TexRel provides rapid training and experimentation, whilst being sufficiently large to avoid overfitting in the context of emergent communications. By comparison with using symbolic inputs, TexRel provides a more realistic alternative whilst remaining efficient and fast to learn. We compare the performance of TexRel with a related relations dataset Shapeworld. We provide baseline performance results on TexRel for sender architectures, receiver architectures and end-to-end architectures. We examine the effect of multitask learning in the context of shapes, colors and relations on accuracy, topological similarity and clustering precision. We investigate whether increasing the size of the latent meaning space improves metrics of compositionality. We carry out a case-study on using TexRel to reproduce the results of an experiment in a recent paper that used symbolic inputs, but using our own non-symbolic inputs, from TexRel, instead.
Implementation of Live Reverse Debugging in LLDB
Venue: arXiv cs.SE 2021
Year: 2021

Debugging is an essential process with a large share of the development effort, being a relentless quest for offensive code through tracing, inspection and iterative running sessions. Probably every developer has been in a situation with a clear wish to rewind time just for a while, only to retry some actions alternatively, instead of restarting the entire session. Well, the genie to fulfill such a wish is known as a reverse debugger. Their inherent technical complexity makes them very hard to implement, while the imposed execution overhead turns them to less preferable for adoption. There are only a few available, most being off-line tools, working on recorded, previously run, sessions. We consider live reverse debuggers both challenging and promising, since they can fit into existing forward debuggers, and we developed the first live reverse debugger on top of LLDB, discussing in detail our implementation approach.
Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we introduce an event-driven trading strategy that predicts stock movements by detecting corporate events from news articles. Unlike existing models that utilize textual features (e.g., bag-of-words) and sentiments to directly make stock predictions, we consider corporate events as the driving force behind stock movements and aim to profit from the temporary stock mispricing that may occur when corporate events take place. The core of the proposed strategy is a bi-level event detection model. The low-level event detector identifies events' existences from each token, while the high-level event detector incorporates the entire article's representation and the low-level detected results to discover events at the article-level. We also develop an elaborately-annotated dataset EDT for corporate event detection and news-based stock prediction benchmark. EDT includes 9721 news articles with token-level event labels as well as 303893 news articles with minute-level timestamps and comprehensive stock price labels. Experiments on EDT indicate that the proposed strategy outperforms all the baselines in winning rate, excess returns over the market, and the average return on each transaction.
Information Harvesting for Far-Field Wireless Power Transfer
Venue: arXiv cs.IT 2021
Year: 2021

Considering ubiquitous connectivity and advanced information processing capability, huge amount of low-power IoT devices are deployed nowadays and the maintenance of those devices which includes firmware/software updates and recharging the units has become a bottleneck for IoT systems. For addressing limited battery constraints, wireless power transfer is a promising approach such that it does not require any physical link between energy harvester and power transfer. Furthermore, combining wireless power transfer with information transmission has become more appealing. In the systems that apply radio signals the wireless power transfer has become a popular trend to harvest RF-radiated energy from received information signal in IoT devices. For those systems, design frameworks mainly deal with the trade-off between information capacity and energy harvesting efficiency. Therein various signaling design frameworks have been proposed for different system preferences between power and information. In addition to this, protecting the information part from potential eavesdropping activity in a service area introduces security considerations for those systems. In this paper, we propose a novel concept, Information Harvesting, for wireless power transfer systems. It introduces a novel protocol design from opposite perspective compared to the existing studies. Particularly, Information Harvesting aims to transmit information through existing wireless power transfer mechanism without interfering/interrupting power transfer.