Reviewerly/exHarmony huggingface dataset:
A Causal Debiasing Framework for Unsupervised Salient Object Detection

Unsupervised Salient Object Detection (USOD) is a promising yet challenging task that aims to learn a salient object detection model without any ground-truth labels. Self-supervised learning based methods have achieved remarkable success recently and have become the dominant approach in USOD. However, we observed that two distribution biases of salient objects limit further performance improvement of the USOD methods, namely, contrast distribution bias and spatial distribution bias. Concretely, contrast distribution bias is essentially a confounder that makes images with similar high-level semantic contrast and/or low-level visual appearance contrast spuriously dependent, thus forming data-rich contrast clusters and leading the training process biased towards the data-rich contrast clusters in the data. Spatial distribution bias means that the position distribution of all salient objects in a dataset is concentrated on the center of the image plane, which could be harmful to off-center objects prediction. This paper proposes a causal based debiasing framework to disentangle the model from the impact of such biases. Specifically, we use causal intervention to perform de-confounded model training to minimize the contrast distribution bias and propose an image-level weighting strategy that softly weights each image's importance according to the spatial distribution bias map. Extensive experiments on 6 benchmark datasets show that our method significantly outperforms previous unsupervised state-of-the-art methods and even surpasses some of the supervised methods, demonstrating our debiasing framework's effectiveness.
Interact, Embed, and EnlargE: Boosting Modality-Specific Representations for Multi-Modal Person Re-identification

Multi-modal person Re-ID introduces more complementary information to assist the traditional Re-ID task. Existing multi-modal methods ignore the importance of modality-specific information in the feature fusion stage. To this end, we propose a novel method to boost modality-specific representations for multi-modal person Re-ID: Interact, Embed, and EnlargE (IEEE). First, we propose a cross-modal interacting module to exchange useful information between different modalities in the feature extraction phase. Second, we propose a relation-based embedding module to enhance the richness of feature descriptors by embedding the global feature into the fine-grained local information. Finally, we propose multi-modal margin loss to force the network to learn modality-specific information for each modality by enlarging the intra-class discrepancy. Superior performance on multi-modal Re-ID dataset RGBNT201 and three constructed Re-ID datasets validate the effectiveness of the proposed method compared with the state-of-the-art approaches.
FFNet: Frequency Fusion Network for Semantic Scene Completion

Semantic scene completion (SSC) requires the estimation of the 3D geometric occupancies of objects in the scene, along with the object categories. Currently, many methods employ RGB-D images to capture the geometric and semantic information of objects. These methods use simple but popular spatial- and channel-wise operations, which fuse the information of RGB and depth data. Yet, they ignore the large discrepancy of RGB-D data and the uncertainty measurements of depth data. To solve this problem, we propose the Frequency Fusion Network (FFNet), a novel method for boosting semantic scene completion by better utilizing RGB-D data. FFNet explicitly correlates the RGB-D data in the frequency domain, different from the features directly extracted by the convolution operation. Then, the network uses the correlated information to guide the feature learning from the RG- B and depth images, respectively. Moreover, FFNet accounts for the properties of different frequency components of RGB- D features. It has a learnable elliptical mask to decompose the features learned from the RGB and depth images, attending to various frequencies to facilitate the correlation process of RGB-D data. We evaluate FFNet intensively on the public SSC benchmarks, where FFNet surpasses the state-of- the-art methods. The code package of FFNet is available at https://github.com/alanWXZ/FFNet.
Indefinite causal order for quantum metrology with quantum thermal noise

A switched quantum channel with indefinite causal order is studied for the fundamental metrological task of phase estimation on a qubit unitary operator affected by quantum thermal noise. Specific capabilities are reported in the switched channel with indefinite order, not accessible with conventional estimation approaches with definite order. Phase estimation can be performed by measuring the control qubit alone, although it does not actively interact with the unitary process – only the probe qubit doing so. Also, phase estimation becomes possible with a fully depolarized input probe or with an input probe aligned with the rotation axis of the unitary, while this is never possible with conventional approaches. The present study extends to thermal noise, investigations previously carried out with the more symmetric and isotropic qubit depolarizing noise, and it contributes to the timely exploration of properties of quantum channels with indefinite causal order relevant to quantum signal and information processing.
FAANet: feature-aligned attention network for real-time multiple object tracking in UAV videos

Multiple object tracking (MOT) in unmanned aerial vehicle (UAV) videos has attracted attention. Because of the observation perspectives of UAV, the object scale changes dramatically and is relatively small. Besides, most MOT algorithms in UAV videos cannot achieve real-time due to the tracking-by-detection paradigm. We propose a feature-aligned attention network (FAANet). It mainly consists of a channel and spatial attention module and a feature-aligned aggregation module. We also improve the real-time performance using the joint-detection-embedding paradigm and structural re-parameterization technique. We validate the effectiveness with extensive experiments on UAV detection and tracking benchmark, achieving new state-of-the-art 44.0 MOTA, 64.6 IDF1 with 38.24 frames per second running speed on a single 1080Ti graphics processing unit.
On Vision Features in Multimodal Machine Translation

Previous work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is on the quality of vision models.In this work, we investigate the impact of vision models on MMT.Given the fact that Transformer is becoming popular in computer vision, we experiment with various strong models (such as Vision Transformer) and enhanced features (such as object-detection and image captioning).We develop a selective attention model to study the patch-level contribution of an image in MMT.On detailed probing tasks, we find that stronger vision models are helpful for learning translation from the visual modality.Our results also suggest the need of carefully examining MMT models, especially when current benchmarks are small-scale and biased.Our code could be found at https: //github.com/libeineu/fairseq_mmt.
An Efficient In-Memory Computing Architecture for Image Enhancement in AI Applications

Random Spray Retinex (RSR) is an effective image enhancement algorithm due to its effectiveness in improving image quality. However, the algorithm computing complexity and the required hardware resources and memory accesses hampered its deployment in many applications scenarios, for instance in IoT systems with limited hardware resources. With the rise of Artificial Intelligence (AI), the use of image enhancement has become essential to improve the performance for many emerging applications. In this paper, we propose the use of the RSR as a pre-processing filter before the task of semantic segmentation of low-quality urban road scenes. Using the publicly available Cityscapes dataset, we compare the performance of a pre-trained deep semantic segmentation network on dark noisy images and on RSR pre-processed images. Our findings confirm the effectiveness of RSR in improving segmentation accuracy. In addition, to address the computation complexity and suitability to edge devices, we propose a novel efficient implementation of the RSR using resistive random access memory (RRAM) technology. The architecture provides highly parallel analog in-memory computing (IMC) capabilities. A detailed, efficient and low latency implementation of the RSR using RRAM-CMOS technology is described. The design is verified using SPICE simulations with measured data from fabricated RRAM and 65nm CMOS technologies. The approach provided here represents an important step towards low-complexity and real-time hardware-friendly architecture and design for Retinex algorithms for edge devices.
GenCompareSum: a hybrid unsupervised summarization method using salience

Text summarization (TS) is an important NLP task.Pre-trained Language Models (PLMs) have been used to improve the performance of TS.However, PLMs are limited by their need of labelled training data and by their attention mechanism, which often makes them unsuitable for use on long documents.To this end, we propose a hybrid, unsupervised, abstractive-extractive approach, in which we walk through a document, generating salient textual fragments representing its key points.We then select the most important sentences of the document by choosing the most similar sentences to the generated texts, calculated using BERTScore.We evaluate the efficacy of generating and using salient textual fragments to guide extractive summarization on documents from the biomedical and general scientific domains.We compare the performance between long and short documents using different generative text models, which are finetuned to generate relevant queries or document titles.We show that our hybrid approach out-performs existing unsupervised methods, as well as state-of-the-art supervised methods, despite not needing a vast amount of labelled training data.
An Efficient Procedure for Removing Salt and Pepper Noise in Images

In this paper, we propose an efficient algorithm for removing salt and pepper noise in images. The process of denoising is implemented in two stages: noise detection followed by noise removal. For noise detection, two extreme intensity values in an image are used to detect possible “noise pixels”. For noise removal, the switching mechanism only selects “noise pixels” for processing to avoid altering any fine image details, and only the identified noise-free pixels are used to achieve better denoising performance. Two filtering techniques, the edge-preserving filtering (EPF) and the extremum-compressing median filtering (ECMF), are employed for edge-preserving and noise removal. The EPF provides higher correlation between the corrupted pixel and neighborhood pixel, which gives rise to better edge preservation. The ECMF can yield an appropriate estimation by selecting the median pixel from the noise-free pixels of current filtering window. The proposed algorithm is tested on different images and provides a better restoration performance over some of the salt and pepper noise filters.
Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Graph neural networks (GNNs) have been a hot spot of recent research and are widely utilized in diverse applications. However, with the use of huger data and deeper models, an urgent demand is unsurprisingly made to accelerate GNNs for more efficient execution. In this paper, we provide a comprehensive survey on acceleration methods for GNNs from an algorithmic perspective. We first present a new taxonomy to classify existing acceleration methods into five categories. Based on the classification, we systematically discuss these methods and highlight their correlations. Next, we provide comparisons from aspects of the efficiency and characteristics of these methods. Finally, we suggest some promising prospects for future research.
Explainable AI-based Federated Deep Reinforcement Learning for Trusted Autonomous Driving

Recently, the concept of autonomous driving became prevalent in the domain of intelligent transportation due to the promises of increased safety, traffic efficiency, fuel economy and reduced travel time. Numerous studies have been conducted in this area to help newcomer vehicles plan their trajectory and velocity. However, most of these proposals only consider trajectory planning using conjunction with a limited data set (i.e., metropolis areas, highways, and residential areas) or assume fully connected and automated vehicle environment. Moreover, these approaches are not explainable and lack trust regarding the contributions of the participating vehicles. To tackle these problems, we design an Explainable Artificial Intelligence (XAI) Federated Deep Reinforcement Learning model to improve the effectiveness and trustworthiness of the trajectory decisions for newcomer Autonomous Vehicles (AVs). When a newcomer AV seeks help for trajectory planning, the edge server launches a federated learning process to train the trajectory and velocity prediction model in a distributed collaborative fashion among participating AVs. One essential challenge in this approach is AVs selection, i.e., how to select the appropriate AVs that should participate in the federated learning process. For this purpose, XAI is first used to compute the contribution of each feature contributed by each vehicle to the overall solution. This helps us compute the trust value for each AV in the model. Then, a trust-based deep reinforcement learning model is put forward to make the selection decisions. Experiments using a real-life dataset show that our solution achieves better performance than benchmark solutions (i.e., Deep Q-Network (DQN), and Random Selection (RS)).
Dataset for pathology reporting of ductal carcinoma <i>in situ</i>, variants of lobular carcinoma <i>in situ</i> and low‐grade lesions: recommendations from the International Collaboration on Cancer Reporting (<scp>ICCR</scp>)

To describe a new international dataset for pathology reporting of ductal carcinoma in situ (DCIS), variants of lobular carcinoma in situ (LCIS) and low-grade lesions (encapsulated papillary carcinoma, solid papillary carcinoma in situ, Paget's disease) produced by the International Collaboration on Cancer Reporting (ICCR).The ICCR, a global alliance of pathology bodies, uses a rigorous and efficient process for the development of evidence-based, structured datasets for pathology reporting of common cancers. Their aim is to support quality pathology reporting and engender understanding between the breast surgeon, pathologist, and oncologist for optimal and uniform patient management globally. Here we describe the dataset for DCIS, some variants of LCIS (namely the pleomorphic and the florid variants), and low-grade lesions by a multidisciplinary panel of internationally recognized experts. The agreed dataset comprises 12 core (required) and five noncore (recommended) elements suitable for both developed and low-income jurisdictions, derived from a review of current evidence. Areas of contention were addressed using a pragmatic approach in the absence of evidence. Use of all core elements is the minimum reporting standard for any individual case. Commentary is provided, explaining each element's clinical relevance, definitions to be applied where appropriate for the agreed list of value options and the rationale for considering the element as core or noncore.This first internationally agreed dataset for DCIS, variants of LCIS, and low-grade lesions reporting will enable their standardization of pathology reporting and enhance clinicopathological communication leading to improved patient outcomes. Widespread adoption will also facilitate international comparisons, multinational clinical trials, and help to improve the management of breast disease globally.
Microseismic event detection in noisy environments with instantaneous spectral Shannon entropy

The detection of information-bearing signals in a time series is very important for describing and analyzing a wide variety of complex physical systems. However, identifying events in low signal-to-noise ratio circumstances remains a challenge once heavy data preprocessing is usually required. In this work, we propose a robust methodology based on the instantaneous-spectral Shannon entropy for capturing microseismic events in noisy environments without the requirement of data preprocessing. We call our proposal the instantaneous spectral entropy detection (ISED) method. We tested the ISED in a couple of real empirical seismic records to detect microseismic events. Our methodology detects microseismic patterns even without any preprocessing, in contrast to usual methods in the literature which need appreciable data preprocessing.
Deep learning based topic and sentiment analysis: COVID19 information seeking on social media

Abstract Social media platforms have become a common place for information exchange among their users. People leave traces of their emotions via text expressions. A systematic collection, analysis, and interpretation of social media data across time and space can give insights into local outbreaks, mental health, and social issues. Such timely insights can help in developing strategies and resources with an appropriate and efficient response. This study analysed a large Spatio-temporal tweet dataset of the Australian sphere related to COVID19. The methodology included a volume analysis, topic modelling, sentiment detection, and semantic brand score to obtain an insight into the COVID19 pandemic outbreak and public discussion in different states and cities of Australia over time. The obtained insights are compared with independently observed phenomena such as government-reported instances.
Pretrained Models for Multilingual Federated Learning

Since the advent of Federated Learning (FL), research has applied these methods to natural language processing (NLP) tasks.Despite a plethora of papers in FL for NLP, no previous works have studied how multilingual text impacts FL algorithms.Furthermore, multilingual text provides an interesting avenue to examine the impact of non-IID text (e.g.different languages) on FL in naturally occurring data.We explore three multilingual language tasks, language modeling, machine translation, and text classification using differing federated and nonfederated learning algorithms.Our results show that using pretrained models reduces the negative effects of FL, helping them to perform near or better than centralized (no privacy) learning, even when using non-IID partitioning. 1
Optimal vocabulary selection approaches for privacy-preserving deep NLP model training for information extraction and cancer epidemiology

BACKGROUND: With the use of artificial intelligence and machine learning techniques for biomedical informatics, security and privacy concerns over the data and subject identities have also become an important issue and essential research topic. Without intentional safeguards, machine learning models may find patterns and features to improve task performance that are associated with private personal information. OBJECTIVE: The privacy vulnerability of deep learning models for information extraction from medical textural contents needs to be quantified since the models are exposed to private health information and personally identifiable information. The objective of the study is to quantify the privacy vulnerability of the deep learning models for natural language processing and explore a proper way of securing patients’ information to mitigate confidentiality breaches. METHODS: The target model is the multitask convolutional neural network for information extraction from cancer pathology reports, where the data for training the model are from multiple state population-based cancer registries. This study proposes the following schemes to collect vocabularies from the cancer pathology reports; (a) words appearing in multiple registries, and (b) words that have higher mutual information. We performed membership inference attacks on the models in high-performance computing environments. RESULTS: The comparison outcomes suggest that the proposed vocabulary selection methods resulted in lower privacy vulnerability while maintaining the same level of clinical task performance.
Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs

Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a three-dimensional (3D) organ mesh for a low-contrast two-dimensional (2D) projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.
HiSA: Hierarchically Semantic Associating for Video Temporal Grounding

Video Temporal Grounding (VTG) aims to locate the time interval in a video that is semantically relevant to a language query. Existing VTG methods interact the query with entangled video features and treat the instances in a dataset independently. However, <i>intra-video entanglement</i> and <i>inter-video connection</i> are rarely considered in these methods, leading to mismatches between the video and language. To this end, we propose a novel method, dubbed Hierarchically Semantic Associating (HiSA), which aims to precisely align the video with language and obtain discriminative representation for further location regression. Specifically, the action factors and background factors are disentangled from adjacent video segments, enforcing precise multimodal interaction and alleviating the intra-video entanglement. In addition, cross-guided contrast is elaborately framed to capture the inter-video connection, which benefits the multimodal understanding to locate the time interval. Extensive experiments on three benchmark datasets demonstrate that our approach significantly outperforms the state-of-the-art methods. The project page is available at: <uri>https://github.com/zhexu1997/HiSA</uri>.