OpenRaiser/Intern-Atlas huggingface dataset:
Are all negatives created equal in contrastive instance discrimination?
Venue: arXiv cs.CV 2020
Year: 2020

Self-supervised learning has recently begun to rival supervised learning on computer vision tasks. Many of the recent approaches have been based on contrastive instance discrimination (CID), in which the network is trained to recognize two augmented versions of the same instance (a query and positive) while discriminating against a pool of other instances (negatives). The learned representation is then used on downstream tasks such as image classification. Using methodology from MoCo v2 (Chen et al., 2020), we divided negatives by their difficulty for a given query and studied which difficulty ranges were most important for learning useful representations. We found a minority of negatives -- the hardest 5% -- were both necessary and sufficient for the downstream task to reach nearly full accuracy. Conversely, the easiest 95% of negatives were unnecessary and insufficient. Moreover, the very hardest 0.1% of negatives were unnecessary and sometimes detrimental. Finally, we studied the properties of negatives that affect their hardness, and found that hard negatives were more semantically similar to the query, and that some negatives were more consistently easy or hard than we would expect by chance. Together, our results indicate that negatives vary in importance and that CID may benefit from more intelligent negative treatment.
Motif Learning in Knowledge Graphs Using Trajectories Of Differential Equations
Venue: arXiv cs.LG 2020
Year: 2020

Knowledge Graph Embeddings (KGEs) have shown promising performance on link prediction tasks by mapping the entities and relations from a knowledge graph into a geometric space (usually a vector space). Ultimately, the plausibility of the predicted links is measured by using a scoring function over the learned embeddings (vectors). Therefore, the capability in preserving graph characteristics including structural aspects and semantics highly depends on the design of the KGE, as well as the inherited abilities from the underlying geometry. Many KGEs use the flat geometry which renders them incapable of preserving complex structures and consequently causes wrong inferences by the models. To address this problem, we propose a neuro differential KGE that embeds nodes of a KG on the trajectories of Ordinary Differential Equations (ODEs). To this end, we represent each relation (edge) in a KG as a vector field on a smooth Riemannian manifold. We specifically parameterize ODEs by a neural network to represent various complex shape manifolds and more importantly complex shape vector fields on the manifold. Therefore, the underlying embedding space is capable of getting various geometric forms to encode complexity in subgraph structures with different motifs. Experiments on synthetic and benchmark dataset as well as social network KGs justify the ODE trajectories as a means to structure preservation and consequently avoiding wrong inferences over state-of-the-art KGE models.
Fault-tolerant and Transactional Stateful Serverless Workflows (extended version)
Venue: arXiv cs.DC 2020
Year: 2020

This paper introduces Beldi, a library and runtime system for writing and composing fault-tolerant and transactional stateful serverless functions. Beldi runs on existing providers and lets developers write complex stateful applications that require fault tolerance and transactional semantics without the need to deal with tasks such as load balancing or maintaining virtual machines. Beldi's contributions include extending the log-based fault-tolerant approach in Olive (OSDI 2016) with new data structures, transaction protocols, function invocations, and garbage collection. They also include adapting the resulting framework to work over a federated environment where each serverless function has sovereignty over its own data. We implement three applications on Beldi, including a movie review service, a travel reservation system, and a social media site. Our evaluation on 1,000 AWS Lambdas shows that Beldi's approach is effective and affordable.
Ferrograph image classification
Venue: arXiv cs.CV 2020
Year: 2020

It has been challenging to identify ferrograph images with a small dataset and various scales of wear particle. A novel model is proposed in this study to cope with these challenging problems. For the problem of insufficient samples, we first proposed a data augmentation algorithm based on the permutation of image patches. Then, an auxiliary loss function of image patch permutation recognition was proposed to identify the image generated by the data augmentation algorithm. Moreover, we designed a feature extraction loss function to force the proposed model to extract more abundant features and to reduce redundant representations. As for the challenge of large change range of wear particle size, we proposed a multi-scale feature extraction block to obtain the multi-scale representations of wear particles. We carried out experiments on a ferrograph image dataset and a mini-CIFAR-10 dataset. Experimental results show that the proposed model can improve the accuracy of the two datasets by 9% and 20% respectively compared with the baseline.
Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion
Venue: arXiv cs.CL 2020
Year: 2020

We show that state-of-the-art self-supervised language models can be readily used to extract relations from a corpus without the need to train a fine-tuned extractive head. We introduce RE-Flex, a simple framework that performs constrained cloze completion over pretrained language models to perform unsupervised relation extraction. RE-Flex uses contextual matching to ensure that language model predictions matches supporting evidence from the input corpus that is relevant to a target relation. We perform an extensive experimental study over multiple relation extraction benchmarks and demonstrate that RE-Flex outperforms competing unsupervised relation extraction methods based on pretrained language models by up to 27.8 $F_1$ points compared to the next-best method. Our results show that constrained inference queries against a language model can enable accurate unsupervised relation extraction.
PP-LinkNet: Improving Semantic Segmentation of High Resolution Satellite Imagery with Multi-stage Training
Venue: arXiv cs.CV 2020
Year: 2020

Road network and building footprint extraction is essential for many applications such as updating maps, traffic regulations, city planning, ride-hailing, disaster response \textit{etc}. Mapping road networks is currently both expensive and labor-intensive. Recently, improvements in image segmentation through the application of deep neural networks has shown promising results in extracting road segments from large scale, high resolution satellite imagery. However, significant challenges remain due to lack of enough labeled training data needed to build models for industry grade applications. In this paper, we propose a two-stage transfer learning technique to improve robustness of semantic segmentation for satellite images that leverages noisy pseudo ground truth masks obtained automatically (without human labor) from crowd-sourced OpenStreetMap (OSM) data. We further propose Pyramid Pooling-LinkNet (PP-LinkNet), an improved deep neural network for segmentation that uses focal loss, poly learning rate, and context module. We demonstrate the strengths of our approach through evaluations done on three popular datasets over two tasks, namely, road extraction and building foot-print detection. Specifically, we obtain 78.19\% meanIoU on SpaceNet building footprint dataset, 67.03\% and 77.11\% on the road topology metric on SpaceNet and DeepGlobe road extraction dataset, respectively.
Online Shaping for ISI Channels with a Limited Number of ADC Bits
Venue: arXiv cs.IT 2020
Year: 2020

An online shaping technique for high performance communication over Gaussian channels with Inter-Symbol Interference (ISI) and receiver Analog to Digital Converter (ADC) noise is presented. The technique uses online transmitter precoding over Pulse Amplitude Modulation (PAM) constellation, designed to shape the symbols distribution so that peak power constraint at the channel output is satisfied. An iterative decoder shares information between a modified M-BCJR module, which computes online the trellis transition probabilities of the shaped distribution, and turbo decoder. The result is a reduction in the required ADC Effective Number Of Bits (ENOB), which is in particular attractive in modern high-speed wireline links. Theoretical bounds are analytically derived which enable to assess the possible gain using shaping. On practical scenarios aim to transmit 200 Gbps and 400 Gbps over printed circuit board, we demonstrate in simulations an overall ENOB gains as high as 1.43 bit and 1.78 bit, respectively, compared to uniform 4-PAM transmission with turbo equalization at the receiver side.
Review the Enterprise Resource Planning in Moroccan Healthcare Organizations
Venue: arXiv cs.CY 2020
Year: 2020

The Hospital Information Systems (HIS) in Morocco take a central place in the process of patient care. An approach is made to analyze the current situation of the HIS within the institutions in order to bring an integral and generic vision, allowing the judicious articulation of the business and IT layers. Currently, the Enterprise Resource Planning (ERP) implemented remains a system consisting of several applications dedicated to specific areas. These systems have become an indispensable element within any hospital. The goal of our study is to discover how the ERP has been used in Moroccan healthcare sector and how these software should be implemented and used to improve healthcare services.
On Minor Left Prime Factorization Problem for Multivariate Polynomial Matrices
Venue: arXiv cs.SC 2020
Year: 2020

A new necessary and sufficient condition for the existence of minor left prime factorizations of multivariate polynomial matrices without full row rank is presented. The key idea is to establish a relationship between a matrix and its full row rank submatrix. Based on the new result, we propose an algorithm for factorizing matrices and have implemented it on the computer algebra system Maple. Two examples are given to illustrate the effectiveness of the algorithm, and experimental data shows that the algorithm is efficient.
Database (Lecture) Streams on the Cloud: An Experience Report on Teaching an Undergrad Database Lecture during a Pandemic
Venue: arXiv cs.CY 2020
Year: 2020

This is an experience report on teaching the undergrad lecture Big Data Engineering at Saarland University in summer term 2020 online. We describe our teaching philosophy, the tools used, what worked and what did not work. As we received extremely positive feedback from the students, in the future, we will continue to use the same teaching model for other lectures.
Better Patch Stitching for Parametric Surface Reconstruction
Venue: arXiv cs.CV 2020
Year: 2020

Recently, parametric mappings have emerged as highly effective surface representations, yielding low reconstruction error. In particular, the latest works represent the target shape as an atlas of multiple mappings, which can closely encode object parts. Atlas representations, however, suffer from one major drawback: The individual mappings are not guaranteed to be consistent, which results in holes in the reconstructed shape or in jagged surface areas. We introduce an approach that explicitly encourages global consistency of the local mappings. To this end, we introduce two novel loss terms. The first term exploits the surface normals and requires that they remain locally consistent when estimated within and across the individual mappings. The second term further encourages better spatial configuration of the mappings by minimizing novel stitching error. We show on standard benchmarks that the use of normal consistency requirement outperforms the baselines quantitatively while enforcing better stitching leads to much better visual quality of the reconstructed objects as compared to the state-of-the-art.
Towards a Policy-as-a-Service Framework to Enable Compliant, Trustworthy AI and HRI Systems in the Wild
Venue: arXiv cs.CY 2020
Year: 2020

Building trustworthy autonomous systems is challenging for many reasons beyond simply trying to engineer agents that 'always do the right thing.' There is a broader context that is often not considered within AI and HRI: that the problem of trustworthiness is inherently socio-technical and ultimately involves a broad set of complex human factors and multidimensional relationships that can arise between agents, humans, organizations, and even governments and legal institutions, each with their own understanding and definitions of trust. This complexity presents a significant barrier to the development of trustworthy AI and HRI systems---while systems developers may desire to have their systems 'always do the right thing,' they generally lack the practical tools and expertise in law, regulation, policy and ethics to ensure this outcome. In this paper, we emphasize the "fuzzy" socio-technical aspects of trustworthiness and the need for their careful consideration during both design and deployment. We hope to contribute to the discussion of trustworthy engineering in AI and HRI by i) describing the policy landscape that must be considered when addressing trustworthy computing and the need for usable trust models, ii) highlighting an opportunity for trustworthy-by-design intervention within the systems engineering process, and iii) introducing the concept of a "policy-as-a-service" (PaaS) framework that can be readily applied by AI systems engineers to address the fuzzy problem of trust during the development and (eventually) runtime process. We envision that the PaaS approach, which offloads the development of policy design parameters and maintenance of policy standards to policy experts, will enable runtime trust capabilities intelligent systems in the wild.
Risk of Interruption of Doctoral Studies and Mental Health in PhD Students
Venue: arXiv cs.CY 2020
Year: 2020

PhD students report a higher prevalence of mental illness symptoms than highly educated individuals in the general population. This situation presents a serious problem for universities. Thus, the knowledge about this phenomenon is of great importance in decision-making. In this paper we use the Nature PhD survey 2019 and estimate several binomial logistic regression models to analyze the risk of interrupting doctoral studies. This risk is measured through the desire of change in either the supervisor or the area of expertise, or the wish of not pursue a PhD. Among the explanatory factors, we focus on the influence of anxiety/depression, discrimination, and bullying. As control variables we use demographic characteristics and others related with the doctoral program. Insufficient contact time with supervisors, and exceeding time spent studying -crossing the 50-h week barrier-, are risk factors of PhD studies interruption, but the most decisive risk factor is poor mental health. Universities should therefore foster an environment of well-being, which allows the development of autonomy and resilience of their PhD students or, when necessary, which fosters the development of conflict resolution skills.
Explainable Recommendations via Attentive Multi-Persona Collaborative Filtering
Venue: arXiv cs.IR 2020
Year: 2020

Two main challenges in recommender systems are modeling users with heterogeneous taste, and providing explainable recommendations. In this paper, we propose the neural Attentive Multi-Persona Collaborative Filtering (AMP-CF) model as a unified solution for both problems. AMP-CF breaks down the user to several latent 'personas' (profiles) that identify and discern the different tastes and inclinations of the user. Then, the revealed personas are used to generate and explain the final recommendation list for the user. AMP-CF models users as an attentive mixture of personas, enabling a dynamic user representation that changes based on the item under consideration. We demonstrate AMP-CF on five collaborative filtering datasets from the domains of movies, music, video games and social networks. As an additional contribution, we propose a novel evaluation scheme for comparing the different items in a recommendation list based on the distance from the underlying distribution of "tastes" in the user's historical items. Experimental results show that AMP-CF is competitive with other state-of-the-art models. Finally, we provide qualitative results to showcase the ability of AMP-CF to explain its recommendations.
A Relaxed Matching Procedure for Unsupervised BLI
Venue: arXiv cs.CL 2020
Year: 2020

Recently unsupervised Bilingual Lexicon Induction (BLI) without any parallel corpus has attracted much research interest. One of the crucial parts in methods for the BLI task is the matching procedure. Previous works impose a too strong constraint on the matching and lead to many counterintuitive translation pairings. Thus, We propose a relaxed matching procedure to find a more precise matching between two languages. We also find that aligning source and target language embedding space bidirectionally will bring significant improvement. We follow the previous iterative framework to conduct experiments. Results on standard benchmark demonstrate the effectiveness of our proposed method, which substantially outperforms previous unsupervised methods.
A lightweight design for serverless Function-as-a-Service
Venue: arXiv cs.DC 2020
Year: 2020

FaaS (Function as a Service) allows developers to upload and execute code in the cloud without managing servers. FaaS offerings from leading public cloud providers are based on system microVM or application container technologies such as Firecracker or Docker. In this paper, we demonstrate that lightweight high-level runtimes, such as WebAssembly, could offer performance and scaling advantages over existing solutions, and could enable finely-grained pay-as-you-use business models. We compared widely used performance benchmarks between Docker native and WebAssembly implementations of the same algorithms. We also discuss the barriers for WebAssembly adoption in serverless computing, such as the lack of tooling support.
HeartBees: Visualizing Crowd Affects
Venue: arXiv cs.HC 2020
Year: 2020

Affective sharing within groups strengthens coordination and empathy, leads to better health outcomes, and increases productivity and performance. Existing tools for affective sharing face one main challenge: creating a representation of collective emotional states that is relatable and universally accessible. To overcome this challenge, we propose HeartBees, a bio-feedback system for visualizing collective emotional states, which maps a multi-dimensional emotion model into a metaphorical visualization of flocks of birds. Grounded on Affective Computing literature and physiological sensing, we mapped physiological indicators that could be obtained from wearable devices into a multi-dimensional emotion model, which, in turn, our HeartBees can make use of. We evaluated our nature-inspired interactive system with 353 online participants, whose responses showed good consensus in the way they subjectively perceived the visualizations. Last, we discuss practical applications of HeartBees.
Data Readiness Report
Venue: arXiv cs.DB 2020
Year: 2020

Data exploration and quality analysis is an important yet tedious process in the AI pipeline. Current practices of data cleaning and data readiness assessment for machine learning tasks are mostly conducted in an arbitrary manner which limits their reuse and results in loss of productivity. We introduce the concept of a Data Readiness Report as an accompanying documentation to a dataset that allows data consumers to get detailed insights into the quality of input data. Data characteristics and challenges on various quality dimensions are identified and documented keeping in mind the principles of transparency and explainability. The Data Readiness Report also serves as a record of all data assessment operations including applied transformations. This provides a detailed lineage for the purpose of data governance and management. In effect, the report captures and documents the actions taken by various personas in a data readiness and assessment workflow. Overtime this becomes a repository of best practices and can potentially drive a recommendation system for building automated data readiness workflows on the lines of AutoML [8]. We anticipate that together with the Datasheets [9], Dataset Nutrition Label [11], FactSheets [1] and Model Cards [15], the Data Readiness Report makes significant progress towards Data and AI lifecycle documentation.
Cost-optimal V2X Service Placement in Distributed Cloud/Edge Environment
Venue: arXiv cs.NI 2020
Year: 2020

Deploying V2X services has become a challenging task. This is mainly due to the fact that such services have strict latency requirements. To meet these requirements, one potential solution is adopting mobile edge computing (MEC). However, this presents new challenges including how to find a cost efficient placement that meets other requirements such as latency. In this work, the problem of cost-optimal V2X service placement (CO-VSP) in a distributed cloud/edge environment is formulated. Additionally, a cost-focused delay-aware V2X service placement (DA-VSP) heuristic algorithm is proposed. Simulation results show that both CO-VSP model and DA-VSP algorithm guarantee the QoS requirements of all such services and illustrates the trade-off between latency and deployment cost.
Self-Supervised Ranking for Representation Learning
Venue: arXiv cs.CV 2020
Year: 2020

We present a new framework for self-supervised representation learning by formulating it as a ranking problem in an image retrieval context on a large number of random views (augmentations) obtained from images. Our work is based on two intuitions: first, a good representation of images must yield a high-quality image ranking in a retrieval task; second, we would expect random views of an image to be ranked closer to a reference view of that image than random views of other images. Hence, we model representation learning as a learning to rank problem for image retrieval. We train a representation encoder by maximizing average precision (AP) for ranking, where random views of an image are considered positively related, and that of the other images considered negatives. The new framework, dubbed S2R2, enables computing a global objective on multiple views, compared to the local objective in the popular contrastive learning framework, which is calculated on pairs of views. In principle, by using a ranking criterion, we eliminate reliance on object-centric curated datasets. When trained on STL10 and MS-COCO, S2R2 outperforms SimCLR and the clustering-based contrastive learning model, SwAV, while being much simpler both conceptually and at implementation. On MS-COCO, S2R2 outperforms both SwAV and SimCLR with a larger margin than on STl10. This indicates that S2R2 is more effective on diverse scenes and could eliminate the need for an object-centric large training dataset for self-supervised representation learning.
Automatic complexity of Fibonacci and Tribonacci words
Venue: arXiv cs.DM 2020
Year: 2020

For a complexity function $C$, the lower and upper $C$-complexity rates of an infinite word $\mathbf{x}$ are \[ \underline{C}(\mathbf x)=\liminf_{n\to\infty} \frac{C(\mathbf{x}\upharpoonright n)}n,\quad \overline{C}(\mathbf x)=\limsup_{n\to\infty} \frac{C(\mathbf{x}\upharpoonright n)}n \] respectively. Here $\mathbf{x}\upharpoonright n$ is the prefix of $x$ of length $n$. We consider the case $C=\mathrm{A_N}$, the nondeterministic automatic complexity. If these rates are strictly between 0 and $1/2$, we call them intermediate. Our main result is that words having intermediate $\mathrm{A_N}$-rates exist, viz. the infinite Fibonacci and Tribonacci words.