OpenRaiser/Intern-Atlas huggingface dataset:
Semantic network analysis of abstract and concrete word associations
Venue: arXiv cs.CE 2021
Year: 2021

In recent years, a new interest for the use of graph-theory based networks has emerged within the field of cognitive science. This has played a key role in mining the large amount of data generated by word association norms. In the present work, we applied semantic network analyses to explore norms of French word associations for concrete and abstract concepts (Lakhzoum et al., 2021). Graph analyses have shown that the network exhibits high clustering coefficient, sparse density, and small average shortest path length for both the concrete and abstract networks. These characteristics are consistent with a small-world structure. Comparisons between local node statistics and global structural topology showed that abstract and concrete concepts present a similar local connectivity but different overall patterns of structural organisation with concrete concepts presenting an organisation in densely connected communities compared to abstract concepts. These patterns confirm previously acquired knowledge about the dichotomy of abstract and concrete concepts on a larger scale. To the best of our knowledge, this is the first attempt to confirm the generalisability of these properties to the French language and with an emphasis on abstract and concrete concepts.
BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation
Venue: arXiv cs.CL 2021
Year: 2021

Natural language processing (NLP) systems are increasingly trained to generate open-ended text rather than classifying between responses. This makes research on evaluation metrics for generated language -- functions that score system output given the context and/or human reference responses -- of critical importance. However, different metrics have different strengths and biases, and reflect human intuitions better on some tasks than others. There is currently no simple, unified way to compare, analyse or evaluate metrics across a representative set of tasks. Here, we describe the Benchmark to Evaluate Automatic Metrics (BEAMetrics), a resource to make research into new metrics itself easier to evaluate. BEAMetrics users can quickly compare existing and new metrics with human judgements across a diverse set of tasks, quality dimensions (fluency vs. coherence vs. informativeness etc), and languages. As generation experts might predict, BEAMetrics reveals stark task-dependent differences between existing metrics, and consistently poor performance on tasks with complex answer spaces or high reliance on general knowledge. While this analysis highlights a critical issue facing current research practice, BEAMetrics also contribute to its resolution by facilitating research into better metrics -- particularly those that can account for the complex interaction between context and general knowledge inherent to many modern NLP applications. BEAMetrics is available under the MIT License: https://github.com/ThomasScialom/BEAMetrics
How to Effectively Identify and Communicate Person-Targeting Media Bias in Daily News Consumption?
Venue: arXiv cs.CY 2021
Year: 2021

Slanted news coverage strongly affects public opinion. This is especially true for coverage on politics and related issues, where studies have shown that bias in the news may influence elections and other collective decisions. Due to its viable importance, news coverage has long been studied in the social sciences, resulting in comprehensive models to describe it and effective yet costly methods to analyze it, such as content analysis. We present an in-progress system for news recommendation that is the first to automate the manual procedure of content analysis to reveal person-targeting biases in news articles reporting on policy issues. In a large-scale user study, we find very promising results regarding this interdisciplinary research direction. Our recommender detects and reveals substantial frames that are actually present in individual news articles. In contrast, prior work rather only facilitates the visibility of biases, e.g., by distinguishing left- and right-wing outlets. Further, our study shows that recommending news articles that differently frame an event significantly improves respondents' awareness of bias.
Enhancing exploration algorithms for navigation with visual SLAM
Venue: arXiv cs.RO 2021
Year: 2021

Exploration is an important step in autonomous navigation of robotic systems. In this paper we introduce a series of enhancements for exploration algorithms in order to use them with vision-based simultaneous localization and mapping (vSLAM) methods. We evaluate developed approaches in photo-realistic simulator in two modes: with ground-truth depths and neural network reconstructed depth maps as vSLAM input. We evaluate standard metrics in order to estimate exploration coverage.
Projected Model Counting: Beyond Independent Support
Venue: arXiv cs.AI 2021
Year: 2021

The past decade has witnessed a surge of interest in practical techniques for projected model counting. Despite significant advancements, however, performance scaling remains the Achilles' heel of this field. A key idea used in modern counters is to count models projected on an \emph{independent support} that is often a small subset of the projection set, i.e. original set of variables on which we wanted to project. While this idea has been effective in scaling performance, the question of whether it can benefit to count models projected on variables beyond the projection set, has not been explored. In this paper, we study this question and show that contrary to intuition, it can be beneficial to project on variables beyond the projection set. In applications such as verification of binarized neural networks, quantification of information flow, reliability of power grids etc., a good upper bound of the projected model count often suffices. We show that in several such cases, we can identify a set of variables, called upper bound support (UBS), that is not necessarily a subset of the projection set, and yet counting models projected on UBS guarantees an upper bound of the true projected model count. Theoretically, a UBS can be exponentially smaller than the smallest independent support. Our experiments show that even otherwise, UBS-based projected counting can be more efficient than independent support-based projected counting, while yielding bounds of very high quality. Based on extensive experiments, we find that UBS-based projected counting can solve many problem instances that are beyond the reach of a state-of-the-art independent support-based projected model counter.
SPON: Enabling Resilient Inter-Ledgers Payments with an Intrusion-Tolerant Overlay
Venue: arXiv cs.CR 2021
Year: 2021

Payment systems are a critical component of everyday life in our society. While in many situations payments are still slow, opaque, siloed, expensive or even fail, users expect them to be fast, transparent, cheap, reliable and global. Recent technologies such as distributed ledgers create opportunities for near-real-time, cheaper and more transparent payments. However, in order to achieve a global payment system, payments should be possible not only within one ledger, but also across different ledgers and geographies. In this paper we propose Secure Payments with Overlay Networks (SPON), a service that enables global payments across multiple ledgers by combining the transaction exchange provided by the Interledger protocol with an intrusion-tolerant overlay of relay nodes to achieve (1) improved payment latency, (2) fault tolerance to benign failures such as node failures and network partitions, and (3) resilience to BGP hijacking attacks. We discuss the design goals and present an implementation based on the Interledger protocol and Spines overlay network. We analyze the resilience of SPON and demonstrate through experimental evaluation that it is able to improve payment latency, recover from path outages, withstand network partition attacks, and disseminate payments fairly across multiple ledgers. We also show how SPON can be deployed to make the communication between different ledgers resilient to BGP hijacking attacks.
Governance and Communication of Algorithmic Decision Making: A Case Study on Public Sector
Venue: arXiv cs.CY 2021
Year: 2021

Algorithmic Decision Making (ADM) has permeated all aspects of society. Government organizations are also affected by this trend. However, the use of ADM has been getting negative attention from the public, media, and interest groups. There is little to no actionable guidelines for government organizations to create positive impact through ADM. In this case study, we examined eight municipal organizations in the Netherlands regarding their actual and intended use of ADM. We interviewed key personnel and decision makers. Our results show that municipalities mostly use ADM in an ad hoc manner, and they have not systematically defined or institutionalized a data science process yet. They operate risk averse, and they clearly express the need for cooperation, guidance, and even supervision at the national level. Third parties, mostly commercial, are often involved in the ADM development lifecycle, without systematic governance. Communication on the use of ADM is generally responsive to negative attention from the media and public. There are strong indications for the need of an ADM governance framework. In this paper, we present our findings in detail, along with actionable insights on governance, communication, and performance evaluation of ADM systems.
Predicting Indian Supreme Court Judgments, Decisions, Or Appeals
Venue: arXiv cs.CY 2021
Year: 2021

Legal predictive models are of enormous interest and value to legal community. The stakeholders, specially, the judges and attorneys can take the best advantages of these models to predict the case outcomes to further augment their future course of actions, for example speeding up the decision making, support the arguments, strengthening the defense, etc. However, accurately predicting the legal decisions and case outcomes is an arduous process, which involves several complex steps -- finding suitable bulk case documents, data extracting, cleansing and engineering, etc. Additionally, the legal complexity further adds to its intricacies. In this paper, we introduce our newly developed ML-enabled legal prediction model and its operational prototype, eLegPredict; which successfully predicts the Indian supreme court decisions. The eLegPredict is trained and tested over 3072 supreme court cases and has achieved 76% accuracy (F1-score). The eLegPredict is equipped with a mechanism to aid end users, where as soon as a document with new case description is dropped into a designated directory, the system quickly reads through its content and generates prediction. To our best understanding, eLegPredict is the first legal prediction model to predict Indian supreme court decisions.
Exploring Individual and Collaborative Storytelling in an Introductory Creative Coding Class
Venue: arXiv cs.CY 2021
Year: 2021

Teaching programming through storytelling is a popular pedagogical approach and an active area of research. However, most previous work in this area focused on K-12 students using block-based programming. Little, if any, work has examined the approach with university students using text-based programming. This experience report fills this gap. Specifically, we report our experience administering three storytelling assignments -- two individual and one collaborative -- in an introductory computer science class with 49 undergraduate students using $\textit{p5.js}$, a text-based programming library for creative coding. Our work contributes an understanding of students' experiences with the three authoring processes and a set of recommendations to improve the administration of and experience with individual and collaborative storytelling with text-based programming.
A Lightweight and Accurate Recognition Framework for Signs of X-ray Weld Images
Venue: arXiv cs.CV 2021
Year: 2021

X-ray images are commonly used to ensure the security of devices in quality inspection industry. The recognition of signs printed on X-ray weld images plays an essential role in digital traceability system of manufacturing industry. However, the scales of objects vary different greatly in weld images, and it hinders us to achieve satisfactory recognition. In this paper, we propose a signs recognition framework based on convolutional neural networks (CNNs) for weld images. The proposed framework firstly contains a shallow classification network for correcting the pose of images. Moreover, we present a novel spatial and channel enhancement (SCE) module to address the above scale problem. This module can integrate multi-scale features and adaptively assign weights for each feature source. Based on SCE module, a narrow network is designed for final weld information recognition. To enhance the practicability of our framework, we carefully design the architecture of framework with a few parameters and computations. Experimental results show that our framework achieves 99.7% accuracy with 1.1 giga floating-point of operations (GFLOPs) on classification stage, and 90.0 mean average precision (mAP) with 176.1 frames per second (FPS) on recognition stage.
Analysis of Indian Agricultural Ecosystem using Knowledge-based Tantra Framework
Venue: arXiv cs.CY 2021
Year: 2021

The information systems have been extremely useful in managing businesses, enterprises, and public institutions such as government departments. But current challenges are increasingly about managing ecosystems. Ecosystem is a useful paradigm to better understand a variety of domains such as biology, business, industry, agriculture, and society. In this paper, we look at the Indian Agricultural ecosystem. It is a mammoth task to assimilate the information for the whole ecosystem consisting of consumers, producers, workers, traders, transporters, industry, and Government. There are myriad interventions by the state and the central Governments, whose efficacy is difficult to track and the outcomes hard to assess. A policy intervention that helps one part of the ecosystem can harm the other. In addition, sustainability and ecological considerations are also extremely important. In this paper, we make use of the Knowledge-based Tantra Social Information Management Framework to analyze the Indian Agricultural Ecosystem and build related Knowledge Graphs. Our analysis spans descriptive, normative, and transformative viewpoints. Tantra Framework makes use of concepts from Zachman Framework to manage aspects of social information through different perspectives and concepts from Unified Foundational Ontology (UFO) to represent interrelationships between aspects.
Contextual Hate Speech Detection in Code Mixed Text using Transformer Based Approaches
Venue: arXiv cs.CL 2021
Year: 2021

In the recent past, social media platforms have helped people in connecting and communicating to a wider audience. But this has also led to a drastic increase in cyberbullying. It is essential to detect and curb hate speech to keep the sanity of social media platforms. Also, code mixed text containing more than one language is frequently used on these platforms. We, therefore, propose automated techniques for hate speech detection in code mixed text from scraped Twitter. We specifically focus on code mixed English-Hindi text and transformer-based approaches. While regular approaches analyze the text independently, we also make use of content text in the form of parent tweets. We try to evaluate the performances of multilingual BERT and Indic-BERT in single-encoder and dual-encoder settings. The first approach is to concatenate the target text and context text using a separator token and get a single representation from the BERT model. The second approach encodes the two texts independently using a dual BERT encoder and the corresponding representations are averaged. We show that the dual-encoder approach using independent representations yields better performance. We also employ simple ensemble methods to further improve the performance. Using these methods we report the best F1 score of 73.07% on the HASOC 2021 ICHCL code mixed data set.
Use and Misuse of the Term Experiment in Mining Software Repositories Research
Venue: arXiv cs.SE 2021
Year: 2021

The significant momentum and importance of Mining Software Repositories (MSR) in Software Engineering (SE) has fostered new opportunities and challenges for extensive empirical research. However, MSR researchers seem to struggle to characterize the empirical methods they use into the existing empirical SE body of knowledge. This is especially the case of MSR experiments. To provide evidence on the special characteristics of MSR experiments and their differences with experiments traditionally acknowledged in SE so far, we elicited the hallmarks that differentiate an experiment from other types of empirical studies and characterized the hallmarks and types of experiments in MSR. We analyzed MSR literature obtained from a small-scale systematic mapping study to assess the use of the term experiment in MSR. We found that 19% of the papers claiming to be an experiment are indeed not an experiment at all but also observational studies, so they use the term in a misleading way. From the remaining 81% of the papers, only one of them refers to a genuine controlled experiment while the others stand for experiments with limited control. MSR researchers tend to overlook such limitations, compromising the interpretation of the results of their studies. We provide recommendations and insights to support the improvement of MSR experiments.
Streaming Machine Learning and Online Active Learning for Automated Visual Inspection
Venue: arXiv cs.CV 2021
Year: 2021

Quality control is a key activity performed by manufacturing companies to verify product conformance to the requirements and specifications. Standardized quality control ensures that all the products are evaluated under the same criteria. The decreased cost of sensors and connectivity enabled an increasing digitalization of manufacturing and provided greater data availability. Such data availability has spurred the development of artificial intelligence models, which allow higher degrees of automation and reduced bias when inspecting the products. Furthermore, the increased speed of inspection reduces overall costs and time required for defect inspection. In this research, we compare five streaming machine learning algorithms applied to visual defect inspection with real-world data provided by Philips Consumer Lifestyle BV. Furthermore, we compare them in a streaming active learning context, which reduces the data labeling effort in a real-world context. Our results show that active learning reduces the data labeling effort by almost 15% on average for the worst case, while keeping an acceptable classification performance. The use of machine learning models for automated visual inspection are expected to speed up the quality inspection up to 40%.
On Predictive Explanation of Data Anomalies
Venue: arXiv cs.LG 2021
Year: 2021

Numerous algorithms have been proposed for detecting anomalies (outliers, novelties) in an unsupervised manner. Unfortunately, it is not trivial, in general, to understand why a given sample (record) is labelled as an anomaly and thus diagnose its root causes. We propose the following reduced-dimensionality, surrogate model approach to explain detector decisions: approximate the detection model with another one that employs only a small subset of features. Subsequently, samples can be visualized in this low-dimensionality space for human understanding. To this end, we develop PROTEUS, an AutoML pipeline to produce the surrogate model, specifically designed for feature selection on imbalanced datasets. The PROTEUS surrogate model can not only explain the training data, but also the out-of-sample (unseen) data. In other words, PROTEUS produces predictive explanations by approximating the decision surface of an unsupervised detector. PROTEUS is designed to return an accurate estimate of out-of-sample predictive performance to serve as a metric of the quality of the approximation. Computational experiments confirm the efficacy of PROTEUS to produce predictive explanations for different families of detectors and to reliably estimate their predictive performance in unseen data. Unlike several ad-hoc feature importance methods, PROTEUS is robust to high-dimensional data.
Unsupervised Finetuning
Venue: arXiv cs.CV 2021
Year: 2021

This paper studies "unsupervised finetuning", the symmetrical problem of the well-known "supervised finetuning". Given a pretrained model and small-scale unlabeled target data, unsupervised finetuning is to adapt the representation pretrained from the source domain to the target domain so that better transfer performance can be obtained. This problem is more challenging than the supervised counterpart, as the low data density in the small-scale target data is not friendly for unsupervised learning, leading to the damage of the pretrained representation and poor representation in the target domain. In this paper, we find the source data is crucial when shifting the finetuning paradigm from supervise to unsupervise, and propose two simple and effective strategies to combine source and target data into unsupervised finetuning: "sparse source data replaying", and "data mixing". The motivation of the former strategy is to add a small portion of source data back to occupy their pretrained representation space and help push the target data to reside in a smaller compact space; and the motivation of the latter strategy is to increase the data density and help learn more compact representation. To demonstrate the effectiveness of our proposed ``unsupervised finetuning'' strategy, we conduct extensive experiments on multiple different target datasets, which show better transfer performance than the naive strategy.
WONDER: Workload Optimized Network Defined Edge Routing
Venue: arXiv cs.NI 2021
Year: 2021

The 5G standards enable cellular network capabilities that significantly improve key network characteristics such as latency, capacity, throughput and reliability, compared to the previous generations of wireless networks. It is, however, clear that in order to achieve these improvements in real network implementations, the supporting physical and logical infrastructure needs to be designed appropriately. The key components of this infrastructure are Radio Access Network, Edge Data Centers, Packet/Optical Interconnection Fabric and Edge Computing. This paper concentrates on the Edge Data Centers, Interconnection and Edge Computing capabilities that target ability to deliver high-performing services on the 5G network by means of intelligent network slicing, traffic routing and coordinated compute workload distribution. We propose new methods of ensuring optimal traffic routing and edge compute workload placement under mobility conditions, subject to application requirements and constraints within a set of interconnected Edge Data Centers, utilizing Segment Routing/IPv6 and software defined control mechanisms.