OpenRaiser/Intern-Atlas huggingface dataset:
GLIME: A new graphical methodology for interpretable model-agnostic explanations
Venue: arXiv cs.LG 2021
Year: 2021

Explainable artificial intelligence (XAI) is an emerging new domain in which a set of processes and tools allow humans to better comprehend the decisions generated by black box models. However, most of the available XAI tools are often limited to simple explanations mainly quantifying the impact of individual features to the models' output. Therefore, human users are not able to understand how the features are related to each other to make predictions, whereas the inner workings of the trained models remain hidden. This paper contributes to the development of a novel graphical explainability tool that not only indicates the significant features of the model but also reveals the conditional relationships between features and the inference capturing both the direct and indirect impact of features to the models' decision. The proposed XAI methodology, termed as gLIME, provides graphical model-agnostic explanations either at the global (for the entire dataset) or the local scale (for specific data points). It relies on a combination of local interpretable model-agnostic explanations (LIME) with graphical least absolute shrinkage and selection operator (GLASSO) producing undirected Gaussian graphical models. Regularization is adopted to shrink small partial correlation coefficients to zero providing sparser and more interpretable graphical explanations. Two well-known classification datasets (BIOPSY and OAI) were selected to confirm the superiority of gLIME over LIME in terms of both robustness and consistency over multiple permutations. Specifically, gLIME accomplished increased stability over the two datasets with respect to features' importance (76%-96% compared to 52%-77% using LIME). gLIME demonstrates a unique potential to extend the functionality of the current state-of-the-art in XAI by providing informative graphically given explanations that could unlock black boxes.
Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer's Disease Prediction
Venue: arXiv cs.CV 2021
Year: 2021

Multimodal neuroimage can provide complementary information about the dementia, but small size of complete multimodal data limits the ability in representation learning. Moreover, the data distribution inconsistency from different modalities may lead to ineffective fusion, which fails to sufficiently explore the intra-modal and inter-modal interactions and compromises the disease diagnosis performance. To solve these problems, we proposed a novel multimodal representation learning and adversarial hypergraph fusion (MRL-AHF) framework for Alzheimer's disease diagnosis using complete trimodal images. First, adversarial strategy and pre-trained model are incorporated into the MRL to extract latent representations from multimodal data. Then two hypergraphs are constructed from the latent representations and the adversarial network based on graph convolution is employed to narrow the distribution difference of hyperedge features. Finally, the hyperedge-invariant features are fused for disease prediction by hyperedge convolution. Experiments on the public Alzheimer's Disease Neuroimaging Initiative(ADNI) database demonstrate that our model achieves superior performance on Alzheimer's disease detection compared with other related models and provides a possible way to understand the underlying mechanisms of disorder's progression by analyzing the abnormal brain connections.
Decidability of Liveness on the TSO Memory Model
Venue: arXiv cs.FL 2021
Year: 2021

An important property of concurrent objects is whether they support progress -a special case of liveness-guarantees, which ensure the termination of individual method calls under system fairness assumptions. Liveness properties have been proposed for concurrent objects. Typical liveness properties includelock-freedom,wait-freedom,deadlock-freedom,starvation-freedom and obstruction-freedom. It is known that the five liveness properties above are decidable on the Sequential Consistency (SC) memory model for a bounded number of processes. However, the problem of decidability of liveness for finite state concurrent programs running on relaxed memory models remains open. In this paper we address this problem for the Total Store Order (TSO) memory model,as found in the x86 architecture. We prove that lock-freedom, wait-freedom,deadlock-freedom and starvation-freedom are undecidable on TSO for a bounded number of processes, while obstruction-freedom is decidable.
Finite-memory strategies in two-player infinite games
Venue: arXiv cs.GT 2021
Year: 2021

We study infinite two-player win/lose games $(A,B,W)$ where $A,B$ are finite and $W \subseteq (A \times B)^ω$. At each round Player 1 and Player 2 concurrently choose one action in $A$ and $B$, respectively. Player 1 wins iff the generated sequence is in $W$. Each history $h \in (A \times B)^*$ induces a game $(A,B,W_h)$ with $W_h := \{ρ\in (A \times B)^ω\mid h ρ\in W\}$. We show the following: if $W$ is in $Δ^0_2$ (for the usual topology), if the inclusion relation induces a well partial order on the $W_h$'s, and if Player 1 has a winning strategy, then she has a finite-memory winning strategy. Our proof relies on inductive descriptions of set complexity, such as the Hausdorff difference hierarchy of the open sets. Examples in $Σ^0_2$ and $Π^0_2$ show some tightness of our result. Our result can be translated to games on finite graphs: e.g. finite-memory determinacy of multi-energy games is a direct corollary, whereas it does not follow from recent general results on finite memory strategies.
SituationCO v1.2's Terms, Properties, Relationships and Axioms -- A Core Ontology for Particular and Generic Situations
Venue: arXiv cs.AI 2021
Year: 2021

The current preprint is an update to SituationCO v1.1 (Situation Core Ontology), which represents its new version 1.2. It specifies and defines all the terms, properties, relationships and axioms of SituationCO v1.2, being an ontology for particular and generic Situations placed at the core level in the context of a four-layered ontological architecture called FCD-OntoArch (Foundational, Core, and Domain Ontological Architecture for Sciences). This is a four-layered ontological architecture, which considers Foundational, Core, Domain and Instance levels. In turn, the domain level is split down in two sub-levels, namely: Top-domain and Low-domain ontological levels. So in fact, we can consider it to be a five-tier architecture. Ontologies at the same level can be related to each other, except for the foundational level where only ThingFO (Thing Foundational Ontology) is found. In addition, ontologies' terms and relationships at lower levels can be semantically enriched by ontologies' terms and relationships from the higher levels. Note that both ThingFO and ontologies at the core level such as SituationCO, ProcessCO, among others, are domain independent. SituationCO's terms and relationships are specialized primarily from ThingFO. It also completely reuses terms primarily from ProcessCO, ProjectCO and GoalCO ontologies. Stereotypes are the used mechanism for enriching SituationCO terms. Note that in the end of this document, we address the SituationCO vs. ThingFO non-taxonomic relationship verification matrix.
Superpixel-guided Iterative Learning from Noisy Labels for Medical Image Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Learning segmentation from noisy labels is an important task for medical image analysis due to the difficulty in acquiring highquality annotations. Most existing methods neglect the pixel correlation and structural prior in segmentation, often producing noisy predictions around object boundaries. To address this, we adopt a superpixel representation and develop a robust iterative learning strategy that combines noise-aware training of segmentation network and noisy label refinement, both guided by the superpixels. This design enables us to exploit the structural constraints in segmentation labels and effectively mitigate the impact of label noise in learning. Experiments on two benchmarks show that our method outperforms recent state-of-the-art approaches, and achieves superior robustness in a wide range of label noises. Code is available at https://github.com/gaozhitong/SP_guided_Noisy_Label_Seg.
Objective video quality metrics application to video codecs comparisons: choosing the best for subjective quality estimation
Venue: arXiv cs.MM 2021
Year: 2021

Quality assessment plays a key role in creating and comparing video compression algorithms. Despite the development of a large number of new methods for assessing quality, generally accepted and well-known codecs comparisons mainly use the classical methods like PSNR, SSIM and new method VMAF. These methods can be calculated following different rules: they can use different frame-by-frame averaging techniques or different summation of color components. In this paper, a fundamental comparison of various versions of generally accepted metrics is carried out to find the most relevant and recommended versions of video quality metrics to be used in codecs comparisons. For comparison, we used a set of videos encoded with video codecs of different standards, and visual quality scores collected for the resulting set of streams since 2018 until 2021
Investigating External Interaction Modality and Design Between Automated Vehicles and Pedestrians at Crossings
Venue: arXiv cs.HC 2021
Year: 2021

In this study, we investigated the effectiveness and user acceptance of three external interaction modalities (i.e., visual, auditory, and visual+auditory) in promoting communications between automated vehicle systems (AVS) and pedestrians at a crosswalk through a large number of combined designs. For this purpose, an online survey was designed and distributed to 68 participants. All participants reported their overall preferences for safety, comfort, trust, ease of understanding, usability, and acceptance towards the systems. Results showed that the visual+auditory interaction modality was the mostly preferred, followed by the visual interaction modality and then the auditory one. We also tested different visual and auditory interaction methods, and found that "Pedestrian silhouette on the front of the vehicle" was the best preferred option while middle-aged participants liked "Chime" much better than young participants though it was overall better preferred than others. Finally, communication between the AVS and pedestrians' phones was not well received due to privacy concerns. These results provided important interface design recommendations in identifying better combination of visual and auditory designs and therefore improving AVS communicating their intention with pedestrians.
Using Adversarial Debiasing to Remove Bias from Word Embeddings
Venue: arXiv cs.CL 2021
Year: 2021

Word Embeddings have been shown to contain the societal biases present in the original corpora. Existing methods to deal with this problem have been shown to only remove superficial biases. The method of Adversarial Debiasing was presumed to be similarly superficial, but this is was not verified in previous works. Using the experiments that demonstrated the shallow removal in other methods, I show results that suggest Adversarial Debiasing is more effective at removing bias and thus motivate further investigation on the utility of Adversarial Debiasing.
A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

We present a survey of ways in which existing scientific knowledge are included when constructing models with neural networks. The inclusion of domain-knowledge is of special interest not just to constructing scientific assistants, but also, many other areas that involve understanding data using human-machine collaboration. In many such instances, machine-based model construction may benefit significantly from being provided with human-knowledge of the domain encoded in a sufficiently precise form. This paper examines the inclusion of domain-knowledge by means of changes to: the input, the loss-function, and the architecture of deep networks. The categorisation is for ease of exposition: in practice we expect a combination of such changes will be employed. In each category, we describe techniques that have been shown to yield significant changes in the performance of deep neural networks.
The Bitlet Model: A Parameterized Analytical Model to Compare PIM and CPU Systems
Venue: arXiv cs.AR 2021
Year: 2021

Nowadays, data-intensive applications are gaining popularity and, together with this trend, processing-in-memory (PIM)-based systems are being given more attention and have become more relevant. This paper describes an analytical modeling tool called Bitlet that can be used, in a parameterized fashion, to estimate the performance and the power/energy of a PIM-based system and thereby assess the affinity of workloads for PIM as opposed to traditional computing. The tool uncovers interesting tradeoffs between, mainly, the PIM computation complexity (cycles required to perform a computation through PIM), the amount of memory used for PIM, the system memory bandwidth, and the data transfer size. Despite its simplicity, the model reveals new insights when applied to real-life examples. The model is demonstrated for several synthetic examples and then applied to explore the influence of different parameters on two systems - IMAGING and FloatPIM. Based on the demonstrations, insights about PIM and its combination with CPU are concluded.
Improving Visualization Interpretation Using Counterfactuals
Venue: arXiv cs.HC 2021
Year: 2021

Complex, high-dimensional data is used in a wide range of domains to explore problems and make decisions. Analysis of high-dimensional data, however, is vulnerable to the hidden influence of confounding variables, especially as users apply ad hoc filtering operations to visualize only specific subsets of an entire dataset. Thus, visual data-driven analysis can mislead users and encourage mistaken assumptions about causality or the strength of relationships between features. This work introduces a novel visual approach designed to reveal the presence of confounding variables via counterfactual possibilities during visual data analysis. It is implemented in CoFact, an interactive visualization prototype that determines and visualizes \textit{counterfactual subsets} to better support user exploration of feature relationships. Using publicly available datasets, we conducted a controlled user study to demonstrate the effectiveness of our approach; the results indicate that users exposed to counterfactual visualizations formed more careful judgments about feature-to-outcome relationships.
Reading Race: AI Recognises Patient's Racial Identity In Medical Images
Venue: arXiv cs.CV 2021
Year: 2021

Background: In medical imaging, prior studies have demonstrated disparate AI performance by race, yet there is no known correlation for race on medical imaging that would be obvious to the human expert interpreting the images. Methods: Using private and public datasets we evaluate: A) performance quantification of deep learning models to detect race from medical images, including the ability of these models to generalize to external environments and across multiple imaging modalities, B) assessment of possible confounding anatomic and phenotype population features, such as disease distribution and body habitus as predictors of race, and C) investigation into the underlying mechanism by which AI models can recognize race. Findings: Standard deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities. Our findings hold under external validation conditions, as well as when models are optimized to perform clinically motivated tasks. We demonstrate this detection is not due to trivial proxies or imaging-related surrogate covariates for race, such as underlying disease distribution. Finally, we show that performance persists over all anatomical regions and frequency spectrum of the images suggesting that mitigation efforts will be challenging and demand further study. Interpretation: We emphasize that model ability to predict self-reported race is itself not the issue of importance. However, our findings that AI can trivially predict self-reported race -- even from corrupted, cropped, and noised medical images -- in a setting where clinical experts cannot, creates an enormous risk for all model deployments in medical imaging: if an AI model secretly used its knowledge of self-reported race to misclassify all Black patients, radiologists would not be able to tell using the same data the model has access to.
On the Use of Time Series Kernel and Dimensionality Reduction to Identify the Acquisition of Antimicrobial Multidrug Resistance in the Intensive Care Unit
Venue: arXiv cs.LG 2021
Year: 2021

The acquisition of Antimicrobial Multidrug Resistance (AMR) in patients admitted to the Intensive Care Units (ICU) is a major global concern. This study analyses data in the form of multivariate time series (MTS) from 3476 patients recorded at the ICU of University Hospital of Fuenlabrada (Madrid) from 2004 to 2020. 18\% of the patients acquired AMR during their stay in the ICU. The goal of this paper is an early prediction of the development of AMR. Towards that end, we leverage the time-series cluster kernel (TCK) to learn similarities between MTS. To evaluate the effectiveness of TCK as a kernel, we applied several dimensionality reduction techniques for visualization and classification tasks. The experimental results show that TCK allows identifying a group of patients that acquire the AMR during the first 48 hours of their ICU stay, and it also provides good classification capabilities.
CogSense: A Cognitively Inspired Framework for Perception Adaptation
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes the CogSense system, which is inspired by sense-making cognition and perception in the mammalian brain to perform perception error detection and perception parameter adaptation using probabilistic signal temporal logic. As a specific application, a contrast-based perception adaption method is presented and validated. The proposed method evaluates perception errors using heterogeneous probe functions computed from the detected objects and subsequently solves a contrast optimization problem to correct perception errors. The CogSense probe functions utilize the characteristics of geometry, dynamics, and detected blob image quality of the objects to develop axioms in a probabilistic signal temporal logic framework. By evaluating these axioms, we can formally verify whether the detections are valid or erroneous. Further, using the CogSense axioms, we generate the probabilistic signal temporal logic-based constraints to finally solve the contrast-based optimization problem to reduce false positives and false negatives.
Copy and Paste method based on Pose for Re-identification
Venue: arXiv cs.CV 2021
Year: 2021

The aim of re-identification is to match objects in surveillance cameras with different viewpoints. Although ReID is developing at a considerably rapid pace, there is currently no processing method for the ReID task in multiple scenarios. However, such processing method is required in real life scenarios, such as those involving security. In the present study, a new ReID scenario was explored, which differs in terms of perspective, background, and pose(walking or cycling). Obviously, ordinary ReID processing methods cannot effectively handle such a scenario, with the introduction of image datasets being the optimal solution, in addition to being considerably expensive. To solve the aforementioned problem, a simple and effective method to generate images in several new scenarios was proposed, which is names the Copy and Paste method based on Pose(CPP). The CPP method is based on key point detection, using copy as paste, to composite a new semantic image dataset in two different semantic image datasets. As an example, pedestrains and bicycles can be used to generate several images that show the same person riding on different bicycles. The CPP method is suitable for ReID tasks in new scenarios and outperforms the traditional methods when applied to the original datasets in original ReID tasks. To be specific, the CPP method can also perform better in terms of generalization for third-party public dataset. The Code and datasets composited by the CPP method will be available in the future.
Geometric Data Augmentation Based on Feature Map Ensemble
Venue: arXiv cs.CV 2021
Year: 2021

Deep convolutional networks have become the mainstream in computer vision applications. Although CNNs have been successful in many computer vision tasks, it is not free from drawbacks. The performance of CNN is dramatically degraded by geometric transformation, such as large rotations. In this paper, we propose a novel CNN architecture that can improve the robustness against geometric transformations without modifying the existing backbones of their CNNs. The key is to enclose the existing backbone with a geometric transformation (and the corresponding reverse transformation) and a feature map ensemble. The proposed method can inherit the strengths of existing CNNs that have been presented so far. Furthermore, the proposed method can be employed in combination with state-of-the-art data augmentation algorithms to improve their performance. We demonstrate the effectiveness of the proposed method using standard datasets such as CIFAR, CUB-200, and Mnist-rot-12k.
Improving the Authentication with Built-in Camera Protocol Using Built-in Motion Sensors: A Deep Learning Solution
Venue: arXiv cs.CR 2021
Year: 2021

We propose an enhanced version of the Authentication with Built-in Camera (ABC) protocol by employing a deep learning solution based on built-in motion sensors. The standard ABC protocol identifies mobile devices based on the photo-response non-uniformity (PRNU) of the camera sensor, while also considering QR-code-based meta-information. During authentication, the user is required to take two photos that contain two QR codes presented on a screen. The presented QR code images also contain a unique probe signal, similar to a camera fingerprint, generated by the protocol. During verification, the server computes the fingerprint of the received photos and authenticates the user if (i) the probe signal is present, (ii) the metadata embedded in the QR codes is correct and (iii) the camera fingerprint is identified correctly. However, the protocol is vulnerable to forgery attacks when the attacker can compute the camera fingerprint from external photos, as shown in our preliminary work. In this context, we propose an enhancement for the ABC protocol based on motion sensor data, as an additional and passive authentication layer. Smartphones can be identified through their motion sensor data, which, unlike photos, is never posted by users on social media platforms, thus being more secure than using photographs alone. To this end, we transform motion signals into embedding vectors produced by deep neural networks, applying Support Vector Machines for the smartphone identification task. Our change to the ABC protocol results in a multi-modal protocol that lowers the false acceptance rate for the attack proposed in our previous work to a percentage as low as 0.07%.