OpenRaiser/Intern-Atlas huggingface dataset:
Combining Counterfactuals With Shapley Values To Explain Image Models
Venue: arXiv cs.LG 2022
Year: 2022

With the widespread use of sophisticated machine learning models in sensitive applications, understanding their decision-making has become an essential task. Models trained on tabular data have witnessed significant progress in explanations of their underlying decision making processes by virtue of having a small number of discrete features. However, applying these methods to high-dimensional inputs such as images is not a trivial task. Images are composed of pixels at an atomic level and do not carry any interpretability by themselves. In this work, we seek to use annotated high-level interpretable features of images to provide explanations. We leverage the Shapley value framework from Game Theory, which has garnered wide acceptance in general XAI problems. By developing a pipeline to generate counterfactuals and subsequently using it to estimate Shapley values, we obtain contrastive and interpretable explanations with strong axiomatic guarantees.
NewsEdits: A News Article Revision Dataset and a Document-Level Reasoning Challenge
Venue: arXiv cs.CL 2022
Year: 2022

News article revision histories provide clues to narrative and factual evolution in news articles. To facilitate analysis of this evolution, we present the first publicly available dataset of news revision histories, NewsEdits. Our dataset is large-scale and multilingual; it contains 1.2 million articles with 4.6 million versions from over 22 English- and French-language newspaper sources based in three countries, spanning 15 years of coverage (2006-2021). We define article-level edit actions: Addition, Deletion, Edit and Refactor, and develop a high-accuracy extraction algorithm to identify these actions. To underscore the factual nature of many edit actions, we conduct analyses showing that added and deleted sentences are more likely to contain updating events, main content and quotes than unchanged sentences. Finally, to explore whether edit actions are predictable, we introduce three novel tasks aimed at predicting actions performed during version updates. We show that these tasks are possible for expert humans but are challenging for large NLP models. We hope this can spur research in narrative framing and help provide predictive tools for journalists chasing breaking news.
A Partially Synchronizing Coloring
Venue: arXiv cs.FL 2022
Year: 2022

Given a finite directed graph, a coloring of its edges turns the graph into a finite-state automaton. A k-synchronizing word of a deterministic automaton is a word in the alphabet of colors at its edges that maps the state set of the automaton at least on k-element subset. A coloring of edges of a directed strongly connected finite graph of a uniform outdegree (constant outdegree of any vertex) is k-synchronizing if the coloring turns the graph into a deterministic finite automaton possessing a k-synchronizing word. For k=1 one has the well known road coloring problem. The recent positive solution of the road coloring problem implies an elegant generalization considered first by Beal and Perrin: a directed finite strongly connected graph of uniform outdegree is k-synchronizing iff the greatest common divisor of lengths of all its cycles is k. Some consequences for coloring of an arbitrary finite digraph are presented. We describe a subquadratic algorithm of the road coloring for the k-synchronization implemented in the package TESTAS. A new linear visualization program demonstrates the obtained coloring. Some consequences for coloring of an arbitrary finite digraph and of such a graph of uniform outdegree are presented.
Integrating deduction and model finding in a language independent setting
Venue: arXiv cs.SE 2022
Year: 2022

Software artifacts are ubiquitous in our lives being an essential part of home appliances, cars, cel phones, and even in more critical activities like aeronautics and health sciences. In this context software failures may produce enormous losses, either economical or, in the extreme, in human lives. Software analysis is an area in software engineering concerned on the application of different techniques in order to prove the (relative) absence of errors in software artifacts. In many cases these methods of analysis are applied by following certain methodological directives that ensure better results. In a previous work we presented the notion of satisfiability calculus as a model theoretical counterpart of Meseguer's proof calculus, providing a formal foundation for a variety of tools that are based on model construction. The present work shows how effective satisfiability sub-calculi, a special type of satisfiability calculi, can be combined with proof calculi, in order to provide foundations to certain methodological approaches to software analysis by relating the construction of finite counterexamples and the absence of proofs, in an abstract categorical setting.
Automated Detection of Typed Links in Issue Trackers
Venue: arXiv cs.SE 2022
Year: 2022

Stakeholders in software projects use issue trackers like JIRA to capture and manage issues, including requirements and bugs. To ease issue navigation and structure project knowledge, stakeholders manually connect issues via links of certain types that reflect different dependencies, such as Epic-, Block-, Duplicate-, or Relate- links. Based on a large dataset of 15 JIRA repositories, we study how well state-of-the-art machine learning models can automatically detect common link types. We found that a pure BERT model trained on titles and descriptions of linked issues significantly outperforms other optimized deep learning models, achieving an encouraging average macro F1-score of 0.64 for detecting 9 popular link types across all repositories (weighted F1-score of 0.73). For the specific Subtask- and Epic- links, the model achieved top F1-scores of 0.89 and 0.97, respectively. Our model does not simply learn the textual similarity of the issues. In general, shorter issue text seems to improve the prediction accuracy with a strong negative correlation of -0.70. We found that Relate-links often get confused with the other links, which suggests that they are likely used as default links in unclear cases. We also observed significant differences across the repositories, depending on how they are used and by whom.
Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework
Venue: arXiv cs.CL 2022
Year: 2022

In this paper we describe our work towards building a generic framework for both multi-modal embedding and multi-label binary classification tasks, while participating in task 5 (Multimedia Automatic Misogyny Identification) of SemEval 2022 competition. Since pretraining deep models from scratch is a resource and data hungry task, our approach is based on three main strategies. We combine different state-of-the-art architectures to capture a wide spectrum of semantic signals from the multi-modal input. We employ a multi-task learning scheme to be able to use multiple datasets from the same knowledge domain to help increase the model's performance. We also use multiple objectives to regularize and fine tune different system components.
Using Machine Learning to Augment Dynamic Time Warping Based Signal Classification
Venue: arXiv cs.LG 2022
Year: 2022

Modern applications such as voice recognition rely on the ability to compare signals to pre-recorded ones to classify them. However, this comparison typically needs to ignore differences due to signal noise, temporal offset, signal magnitude, and other external factors. The Dynamic Time Warping (DTW) algorithm quantifies this similarity by finding corresponding regions between the signals and non-linearly warping one signal by stretching and shrinking it. Unfortunately, searching through all "warps" of a signal to find the best corresponding regions is computationally expensive. The FastDTW algorithm improves performance, but sacrifices accuracy by only considering small signal warps. My goal is to improve the speed of DTW while maintaining high accuracy. My key insight is that in any particular application domain, signals exhibit specific types of variation. For example, the accelerometer signal measured for two different people would differ based on their stride length and weight. My system, called Machine Learning DTW (MLDTW), uses machine learning to learn the types of warps that are common in a particular domain. It then uses the learned model to improve DTW performance by limiting the search of potential warps appropriately. My results show that compared to FastDTW, MLDTW is at least as fast and reduces errors by 60% on average across four different data sets. These improvements will significantly impact a wide variety of applications (e.g. health monitoring) and enable more scalable processing of multivariate, higher frequency, and longer signal recordings.
An autonomous robot for pruning modern, planar fruit trees
Venue: arXiv cs.RO 2022
Year: 2022

Dormant pruning of fruit trees is an important task for maintaining tree health and ensuring high-quality fruit. Due to decreasing labor availability, pruning is a prime candidate for robotic automation. However, pruning also represents a uniquely difficult problem for robots, requiring robust systems for perception, pruning point determination, and manipulation that must operate under variable lighting conditions and in complex, highly unstructured environments. In this paper, we introduce a system for pruning sweet cherry trees (in a planar tree architecture called an upright fruiting offshoot configuration) that integrates various subsystems from our previous work on perception and manipulation. The resulting system is capable of operating completely autonomously and requires minimal control of the environment. We validate the performance of our system through field trials in a sweet cherry orchard, ultimately achieving a cutting success rate of 58%. Though not fully robust and requiring improvements in throughput, our system is the first to operate on fruit trees and represents a useful base platform to be improved in the future.
Attributions Beyond Neural Networks: The Linear Program Case
Venue: arXiv cs.LG 2022
Year: 2022

Linear Programs (LPs) have been one of the building blocks in machine learning and have championed recent strides in differentiable optimizers for learning systems. While there exist solvers for even high-dimensional LPs, understanding said high-dimensional solutions poses an orthogonal and unresolved problem. We introduce an approach where we consider neural encodings for LPs that justify the application of attribution methods from explainable artificial intelligence (XAI) designed for neural learning systems. The several encoding functions we propose take into account aspects such as feasibility of the decision space, the cost attached to each input, or the distance to special points of interest. We investigate the mathematical consequences of several XAI methods on said neural LP encodings. We empirically show that the attribution methods Saliency and LIME reveal indistinguishable results up to perturbation levels, and we propose the property of Directedness as the main discriminative criterion between Saliency and LIME on one hand, and a perturbation-based Feature Permutation approach on the other hand. Directedness indicates whether an attribution method gives feature attributions with respect to an increase of that feature. We further notice the baseline selection problem beyond the classical computer vision setting for Integrated Gradients.
Resilience and Energy-Awareness in Constraint-Driven-Controlled Multi-Robot Systems
Venue: arXiv cs.RO 2022
Year: 2022

In the context of constraint-driven control of multi-robot systems, in this paper, we propose an optimization-based framework that is able to ensure resilience and energy-awareness of teams of robots. The approach is based on a novel, frame-theoretic, measure of resilience which allows us to analyze and enforce resilient behaviors of multi-robot systems. The properties of resilience and energy-awareness are encoded as constraints of a convex optimization program which is used to synthesize the robot control inputs. This allows for the combination of such properties with the execution of coordinated tasks to achieve resilient and energy-aware robot operations. The effectiveness of the proposed method is illustrated in a simulated scenario where a team of robots is deployed to execute two tasks subject to energy and resilience constraints.
Deployment of AGRI-BOT in Greenhouse Administration
Venue: arXiv cs.RO 2022
Year: 2022

Modern agriculture is constantly evolving to increase production despite unfavorable environmental conditions. A promising approach is 'greenhouse cultivation' providing a microclimate to the cultivated plants to overcome unfavorable climate. However, massive-sized greenhouses develop non-uniform micro-climate throughout the complex requiring high degree of human supervision. We propose deploying an Agri-Bot to create and maintain positive ecological conditions in the greenhouse, reducing labor costs and increasing production. The prototype will contain two primary systems, the navigation system and the data analytics system. The navigation system will be controlled by an Arduino, and data analytics will be handled using an ESP8266 microchip. Numerous sensors for measuring the greenhouse parameters will be mounted on the robot. It will follow a predefined path, while taking readings at checkpoints. The microchip will collect and process data from sensors, transmit to the cloud, and give commands to the actuators. The soil and climate parameters like temperature, humidity, light intensity, soil moisture, pH will be measured periodically. When the parameters are not within a specified range, the Agri-Bot will take corrective actions like switching on blowers/heaters, starting irrigation etc. If external intervention is required, eg., fertilizer, it will indicate accordingly. Deploying such an Agri-Bot for monitoring and controlling microclimate in large-scale greenhouses can mitigate labor costs while increasing productivity. In spite of an initial cost, it can provide a high return on investment by providing flexibility, low power consumption and easy management to help greenhouse be water efficient, provide evenly dispersed and controlled sunlight intensity, temperature and humidity.
Machine vision for vial positioning detection toward the safe automation of material synthesis
Venue: arXiv cs.CV 2022
Year: 2022

Although robot-based automation in chemistry laboratories can accelerate the material development process, surveillance-free environments may lead to dangerous accidents primarily due to machine control errors. Object detection techniques can play vital roles in addressing these safety issues; however, state-of-the-art detectors, including single-shot detector (SSD) models, suffer from insufficient accuracy in environments involving complex and noisy scenes. With the aim of improving safety in a surveillance-free laboratory, we report a novel deep learning (DL)-based object detector, namely, DenseSSD. For the foremost and frequent problem of detecting vial positions, DenseSSD achieved a mean average precision (mAP) over 95% based on a complex dataset involving both empty and solution-filled vials, greatly exceeding those of conventional detectors; such high precision is critical to minimizing failure-induced accidents. Additionally, DenseSSD was observed to be highly insensitive to the environmental changes, maintaining its high precision under the variations of solution colors or testing view angles. The robustness of DenseSSD would allow the utilized equipment settings to be more flexible. This work demonstrates that DenseSSD is useful for enhancing safety in an automated material synthesis environment, and it can be extended to various applications where high detection accuracy and speed are both needed.
Faster Decomposition of Weighted Graphs into Cliques using Fisher's Inequality
Venue: arXiv cs.DS 2022
Year: 2022

Mining groups of genes that consistently co-express is an important problem in biomedical research, where it is critical for applications such as drug-repositioning and designing new disease treatments. Recently, Cooley et al. modeled this problem as Exact Weighted Clique Decomposition (EWCD) in which, given an edge-weighted graph $G$ and a positive integer $k$, the goal is to decompose $G$ into at most $k$ (overlapping) weighted cliques so that an edge's weight is exactly equal to the sum of weights for cliques it participates in. They show EWCD is fixed-parameter-tractable, giving a $4^k$-kernel alongside a backtracking algorithm (together called cricca) to iteratively build a decomposition. Unfortunately, because of inherent exponential growth in the space of potential solutions, cricca is typically able to decompose graphs only when $k \leq 11$. In this work, we establish reduction rules that exponentially decrease the size of the kernel (from $4^k$ to $k2^k$) for EWCD. In addition, we use insights about the structure of potential solutions to give new search rules that speed up the decomposition algorithm. At the core of our techniques is a result from combinatorial design theory called Fisher's inequality characterizing set systems with restricted intersections. We deploy our kernelization and decomposition algorithms (together called DeCAF) on a corpus of biologically-inspired data and obtain over two orders of magnitude speed-up over cricca. As a result, DeCAF scales to instances with $k \geq 17$.
CMNEROne at SemEval-2022 Task 11: Code-Mixed Named Entity Recognition by leveraging multilingual data
Venue: arXiv cs.CL 2022
Year: 2022

Identifying named entities is, in general, a practical and challenging task in the field of Natural Language Processing. Named Entity Recognition on the code-mixed text is further challenging due to the linguistic complexity resulting from the nature of the mixing. This paper addresses the submission of team CMNEROne to the SEMEVAL 2022 shared task 11 MultiCoNER. The Code-mixed NER task aimed to identify named entities on the code-mixed dataset. Our work consists of Named Entity Recognition (NER) on the code-mixed dataset by leveraging the multilingual data. We achieved a weighted average F1 score of 0.7044, i.e., 6% greater than the baseline.
Toward the smooth mesh climbing of a miniature robot using bioinspired soft and expandable claws
Venue: arXiv cs.RO 2022
Year: 2022

While most micro-robots face difficulty traveling on rugged and uneven terrain, beetles can walk smoothly on the complex substrate without slipping or getting stuck on the surface due to their stiffness-variable tarsi and expandable hooks on the tip of tarsi. In this study, we found that beetles actively bent and expanded their claws regularly to crawl freely on mesh surfaces. Inspired by the crawling mechanism of the beetles, we designed an 8-cm miniature climbing robot equipping artificial claws to open and bend in the same cyclic manner as natural beetles. The robot can climb freely with a controllable gait on the mesh surface, steep incline of the angle of 60°, and even transition surface. To our best knowledge, this is the first micro-scale robot that can climb both the mesh surface and cliffy incline.
Hardening DNNs against Transfer Attacks during Network Compression using Greedy Adversarial Pruning
Venue: arXiv cs.LG 2022
Year: 2022

The prevalence and success of Deep Neural Network (DNN) applications in recent years have motivated research on DNN compression, such as pruning and quantization. These techniques accelerate model inference, reduce power consumption, and reduce the size and complexity of the hardware necessary to run DNNs, all with little to no loss in accuracy. However, since DNNs are vulnerable to adversarial inputs, it is important to consider the relationship between compression and adversarial robustness. In this work, we investigate the adversarial robustness of models produced by several irregular pruning schemes and by 8-bit quantization. Additionally, while conventional pruning removes the least important parameters in a DNN, we investigate the effect of an unconventional pruning method: removing the most important model parameters based on the gradient on adversarial inputs. We call this method Greedy Adversarial Pruning (GAP) and we find that this pruning method results in models that are resistant to transfer attacks from their uncompressed counterparts.
Conformance Testing of Mealy Machines Under Input Restrictions
Venue: arXiv cs.FL 2022
Year: 2022

We introduce a grey-box conformance testing method for networks of interconnected Mealy Machines. This approach addresses the scenario where all interfaces of the component under test are observable, but its inputs are under the control of other white-box components. We prove new conditions for full fault detection that exploit repetitions across branching executions of the composite machine in a novel way.Finally, we provide experimental evaluation of our approach on cascade compositions of up to a thousand states, and show that it notably out-performs existing black-box testing techniques.
Predicting Gender via Eye Movements
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we report the first stable results on gender prediction via eye movements. We use a dataset with images of faces as stimuli and with a large number of 370 participants. Stability has two meanings for us: first that we are able to estimate the standard deviation (SD) of a single prediction experiment (it is around 4.1 %); this is achieved by varying the number of participants. And second, we are able to provide a mean accuracy with a very low standard error (SEM): our accuracy is 65.2 %, and the SEM is 0.80 %; this is achieved through many runs of randomly selecting training and test sets for the prediction. Our study shows that two particular classifiers achieve the best accuracies: Random Forests and Logistic Regression. Our results reconfirm previous findings that females are more biased towards the left eyes of the stimuli.
PolyU-BPCoMa: A Dataset and Benchmark Towards Mobile Colorized Mapping Using a Backpack Multisensorial System
Venue: arXiv cs.CV 2022
Year: 2022

Constructing colorized point clouds from mobile laser scanning and images is a fundamental work in surveying and mapping. It is also an essential prerequisite for building digital twins for smart cities. However, existing public datasets are either in relatively small scales or lack accurate geometrical and color ground truth. This paper documents a multisensorial dataset named PolyU-BPCoMA which is distinctively positioned towards mobile colorized mapping. The dataset incorporates resources of 3D LiDAR, spherical imaging, GNSS and IMU on a backpack platform. Color checker boards are pasted in each surveyed area as targets and ground truth data are collected by an advanced terrestrial laser scanner (TLS). 3D geometrical and color information can be recovered in the colorized point clouds produced by the backpack system and the TLS, respectively. Accordingly, we provide an opportunity to benchmark the mapping and colorization accuracy simultaneously for a mobile multisensorial system. The dataset is approximately 800 GB in size covering both indoor and outdoor environments. The dataset and development kits are available at https://github.com/chenpengxin/PolyU-BPCoMa.git.