Reviewerly/exHarmony huggingface dataset:
ARF-Net: An Adaptive Receptive Field Network for breast mass segmentation in whole mammograms and ultrasound images

UNet adopting an encoder-decoder structure has been used widely in medical image segmentation tasks for its outstanding performance. However, in our work, we find that UNet has the worse segmentation performance of small masses. The reason behind this is that the sizes of receptive fields are limited. In this work, to address this issue, we develop a novel end-to-end model, Adaptive Receptive Field Network (ARF-Net), for the precise breast mass segmentation in whole mammographic images and ultrasound images. ARF-Net composes of an encoder network and a corresponding decoder network, followed by a pixel-wise classifier. In ARF-Net, a Selective Receptive Filed Module (SRFM) is proposed to allocate the suitable sizes of receptive fields to the breast masses of different sizes. SRFM consists of a Multiple Receptive Field Module (MRFM) for generating multiple receptive fields of different sizes and a Multi-Scale Selection Module (MSSM) for selecting the suitable sizes of receptive fields based on the objects’ size. The proposed ARF-Net achieves the dice index of 86.1%,85.75%, and 88.12% on the two mammographic databases (INbreast and CBIS-DDSM) and one ultrasonic database (UDIAT), respectively. Moreover, extensive ablation experiments show that ARF-Net transcends several state-of-the-art segmentation networks, and the developed MSSM exceeds several counterparts.
Hierarchical attentive knowledge graph embedding for personalized recommendation

Knowledge graphs (KGs) have proven to be effective for high-quality recommendation, where the connectivities between users and items provide rich and complementary information to user-item interactions. Most existing methods, however, are insufficient to exploit the KGs for capturing user preferences, as they either represent the user-item connectivities via paths with limited expressiveness or implicitly model them by propagating information over the entire KG with inevitable noise. In this paper, we design a novel hierarchical attentive knowledge graph embedding (HAKG) framework to exploit the KGs for effective recommendation. Specifically, HAKG first extracts the expressive subgraphs that link user-item pairs to characterize their connectivities, which accommodate both the semantics and topology of KGs. The subgraphs are then encoded via a hierarchical attentive subgraph encoding to generate effective subgraph embeddings for enhanced user preference prediction. Extensive experiments show the superiority of HAKG against state-of-the-art recommendation methods, as well as its potential in alleviating the data sparsity issue.
Capturing the objects of vision with neural networks

Human visual perception carves a scene at its physical joints, decomposing the world into objects, which are selectively attended, tracked and predicted as we engage our surroundings. Object representations emancipate perception from the sensory input, enabling us to keep in mind that which is out of sight and to use perceptual content as a basis for action and symbolic cognition. Human behavioural studies have documented how object representations emerge through grouping, amodal completion, proto-objects and object files. By contrast, deep neural network models of visual object recognition remain largely tethered to sensory input, despite achieving human-level performance at labelling objects. Here, we review related work in both fields and examine how these fields can help each other. The cognitive literature provides a starting point for the development of new experimental tasks that reveal mechanisms of human object perception and serve as benchmarks driving the development of deep neural network models that will put the object into object recognition.
Implicit Deep Learning

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 7 August 2020Accepted: 18 June 2021Published online: 16 September 2021Keywordsdeep learning, deep equilibrium models, Perron--Frobenius theory, fixed-point equations, robustness, adversarial attacksAMS Subject Headings690C26, 49M99, 65K10, 62M45, 26B10Publication DataISSN (online): 2577-0187Publisher: Society for Industrial and Applied MathematicsCODEN: sjmdaq
Quantum Neural Networks: Concepts, Applications, and Challenges

Quantum deep learning is a research field for the use of quantum computing techniques for training deep neural networks. The research topics and directions of deep learning and quantum computing have been separated for long time, however by discovering that quantum circuits can act like artificial neural networks, quantum deep learning research is widely adopted. This paper explains the backgrounds and basic principles of quantum deep learning and also introduces major achievements. After that, this paper discusses the challenges of quantum deep learning research in multiple perspectives. Lastly, this paper presents various future research directions and application fields of quantum deep learning.
Digital competence and online language teaching: Hong Kong language teacher practices in primary and secondary classrooms

Hong Kong was one of the first public school systems in the world to shift to online learning in response to the COVID-19 pandemic. Primary and secondary school language teachers had to adapt instruction to cater to their students' diverse linguistic and educational needs. Drawing from the European Framework for the Digital Competence of Educators (DCE), the current study examines the experiences of primary and secondary school English language teachers' digital competence as they navigated digital resources, teaching and learning, assessment, and empowering learners in an emergency remote teaching context. Using a two-stage design, this study drew from qualitative survey data (N = 73) and follow-up interviews (N = 10). Findings illuminate how language teachers responded to their students' educational and linguistic needs in online environments. Notably, educators adapted their virtual instruction to cater to students with varying English language proficiency levels and leveraged both synchronous and asynchronous platforms to best support student learning. Moreover, the current study sheds light on how the DCE framework might be used as a tool to assess the digital competencies of educators and better prepare them for virtual teaching. It concludes with a call for more research on emergency remote teaching, particularly within primary and secondary language classrooms.
LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis

Recent advances in document image analysis (DIA) have been primarily driven by the application of neural networks. Ideally, research outcomes could be easily deployed in production and extended for further investigation. However, various factors like loosely organized codebases and sophisticated model configurations complicate the easy reuse of important innovations by a wide audience. Though there have been on-going efforts to improve reusability and simplify deep learning (DL) model development in disciplines like natural language processing and computer vision, none of them are optimized for challenges in the domain of DIA. This represents a major gap in the existing toolkit, as DIA is central to academic research across a wide range of disciplines in the social sciences and humanities. This paper introduces LayoutParser, an open-source library for streamlining the usage of DL in DIA research and applications. The core LayoutParser library comes with a set of simple and intuitive interfaces for applying and customizing DL models for layout detection, character recognition, and many other document processing tasks. To promote extensibility, LayoutParser also incorporates a community platform for sharing both pre-trained models and full document digitization pipelines. We demonstrate that LayoutParser is helpful for both lightweight and large-scale digitization pipelines in real-word use cases. The library is publicly available at https://layout-parser.github.io.
Federated Learning for Non-IID Data via Unified Feature Learning and Optimization Objective Alignment

Federated Learning (FL) aims to establish a shared model across decentralized clients under the privacy-preserving constraint. Despite certain success, it is still challenging for FL to deal with non-IID (non-independent and identical distribution) client data, which is a general scenario in real-world FL tasks. It has been demonstrated that the performance of FL will be reduced greatly under the non-IID scenario, since the discrepant data distributions will induce optimization inconsistency and feature divergence issues. Besides, naively minimizing an aggregate loss function in this scenario may have negative impacts on some clients and thus deteriorate their personal model performance. To address these issues, we propose a Unified Feature learning and Optimization objectives alignment method (FedUFO) for non-IID FL. In particular, an adversary module is proposed to reduce the divergence on feature representation among different clients, and two consensus losses are proposed to reduce the inconsistency on optimization objectives from two perspectives. Extensive experiments demonstrate that our FedUFO can outperform the state-of-the-art approaches, including the competitive one data-sharing method. Besides, FedUFO can enable more reasonable and balanced model performance among different clients.
Deep Reparametrization of Multi-Frame Super-Resolution and Denoising

We propose a deep reparametrization of the maximum a posteriori formulation commonly employed in multi-frame image restoration tasks. Our approach is derived by introducing a learned error metric and a latent representation of the target image, which transforms the MAP objective to a deep feature space. The deep reparametrization allows us to directly model the image formation process in the latent space, and to integrate learned image priors into the prediction. Our approach thereby leverages the advantages of deep learning, while also benefiting from the principled multi-frame fusion provided by the classical MAP formulation. We validate our approach through comprehensive experiments on burst denoising and burst super-resolution datasets. Our approach sets a new state-of-the-art for both tasks, demonstrating the generality and effectiveness of the proposed formulation.
MedShare: A Privacy-Preserving Medical Data Sharing System by Using Blockchain

Electronic Health Record (EHR) and its privacy have attracted widespread attention with the development of the healthcare industry in recent years. As locking medical data in a single healthcare center causes information isolation, healthcare centers are motivated to build medical data sharing systems. However, existing systems highly rely on the trusted centralized servers, which are vulnerable to distributed denial of service (DDoS) attacks and the single point of failure. Moreover, it is a non-trivial matter to authorize multiple users to search and access EHR in a privacy-preserving manner. In this paper, we propose MedShare, a decentralized framework for secure EHR sharing. Our design utilizes the smart contract technique of blockchain to establish a trusted platform for healthcare centers to share their encrypted EHR. Considering that fine-grained access control is essential in practical EHR sharing service, we devise a constant-size attribute-based encryption (ABE) scheme, where the access policy is embedded in search result on the blockchain. Besides, we propose an efficient scheme that enables authorized MedShare users to perform multi-keyword boolean search operations over encrypted EHR. We formally analyze the security strengths and implement the system prototype on Ethereum. Evaluation results demonstrate that MedShare is efficient for EHR sharing.
Interpretable Image Recognition by Constructing Transparent Embedding Space

Humans usually explain their reasoning (e.g. classification) by dissecting the image and pointing out the evidence from these parts to the concepts in their minds. Inspired by this cognitive process, several part-level interpretable neural network architectures have been proposed to explain the predictions. However, they suffer from the complex data structure and confusing the effect of the individual part to output category. In this work, an interpretable image recognition deep network is designed by introducing a plug-in transparent embedding space (TesNet) to bridge the high-level input patches (e.g. CNN feature maps) and the out- put categories. This plug-in embedding space is spanned by transparent basis concepts which are constructed on the Grassmann manifold. These basis concepts are enforced to be category-aware and within-category concepts are orthogonal to each other, which makes sure the embedding space is disentangled. Meanwhile, each basis concept can be traced back to the particular image patches, thus they are transparent and friendly to explain the reasoning process. By comparing with state-of-the-art interpretable methods, TesNet is much more beneficial to classification tasks, esp. providing better interpretability on predictions and improve the final accuracy. The code is available at https://github.com/JackeyWang96/TesNet.
MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions

Spatio-temporal action detection is an important and challenging problem in video understanding. The existing action detection benchmarks are limited in aspects of small numbers of instances in a trimmed video or low-level atomic actions. This paper aims to present a new multi-person dataset of spatio-temporal localized sports actions, coined as MultiSports. We first analyze the important ingredients of constructing a realistic and challenging dataset for spatio-temporal action detection by proposing three criteria: (1) multi-person scenes and motion dependent identification, (2) with well-defined boundaries, (3) relatively fine-grained classes of high complexity. Based on these guide-lines, we build the dataset of MultiSports v1.0 by selecting 4 sports classes, collecting 3200 video clips, and annotating 37701 action instances with 902k bounding boxes. Our datasets are characterized with important properties of high diversity, dense annotation, and high quality. Our Multi-Sports, with its realistic setting and detailed annotations, exposes the intrinsic challenges of spatio-temporal action detection. To benchmark this, we adapt several baseline methods to our dataset and give an indepth analysis on the action detection results in our dataset. We hope our MultiSports can serve as a standard benchmark for spatio-temporal action detection in the future. Our dataset website is at https://deeperaction.github.io/multisports/.
A deep learning semantic template matching framework for remote sensing image registration

We propose a deep learning framework by the probability of the predicting semantic spatial position distribution for remote sensing image registration. Traditional matching methods optimize similarity metrics with pixel-by-pixel searching, which is time consuming and sensitive to radiometric differences. Driven by learning-based methods, we take the reference and template images as inputs and map them to the semantic distribution position of the corresponding reference image. We apply the affine invariant to obtain a correspondence between the overlap of the barycenter position of the semantic template and the center pixel point, which transforms the semantic boundary alignment into a point-to-point matching problem. Additionally, two loss functions are proposed, one for optimizing the subpixel matching position and the other for determining the semantic spatial probability distribution of the matching template. In this work, we explore the influence of the template radius size, the filling form of training labels, and the weighted combination of loss function on the matching accuracy. Our experiments show that the trained model is robust to template deformation while still operating orders of magnitude faster. Furthermore, our proposed method implements high matching accuracy in four large scene images with radiometric differences. This ensures the improved speed of remote sensing image analysis and pipeline processing while facilitating novel directions in learning-based registration. Our code is freely available at https://github.com/liliangzhi110/semantictemplatematching.
Online Refinement of Low-level Feature Based Activation Map for Weakly Supervised Object Localization

We present a two-stage learning framework for weakly supervised object localization (WSOL). While most previous efforts rely on high-level feature based CAMs (Class Activation Maps), this paper proposes to localize objects using the low-level feature based activation maps. In the first stage, an activation map generator produces activation maps based on the low-level feature maps in the classifier, such that rich contextual object information is included in an online manner. In the second stage, we employ an evaluator to evaluate the activation maps predicted by the activation map generator. Based on this, we further propose a weighted entropy loss, an attentive erasing, and an area loss to drive the activation map generator to substantially reduce the uncertainty of activations between object and background, and explore less discriminative regions. Based on the low-level object information preserved in the first stage, the second stage model gradually generates a well-separated, complete, and compact activation map of object in the image, which can be easily thresholded for accurate localization. Extensive experiments on CUB-200-2011 and ImageNet-1K datasets show that our framework surpasses previous methods by a large margin, which sets a new state-of-the-art for WSOL. Code will be available soon.
Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans

This paper introduces a pipeline to parametrically sample and render static multi-task vision datasets from comprehensive 3D scans from the real-world. In addition to enabling interesting lines of research, we show the tooling and generated data suffice to train robust vision models. Familiar architectures trained on a generated starter dataset reached state-of-the-art performance on multiple common vision tasks and benchmarks, despite having seen no benchmark or non-pipeline data. The depth estimation network outperforms MiDaS and the surface normal estimation network is the first to achieve human-level performance for in-the-wild surface normal estimation—at least according to one metric on the OASIS benchmark. The Dockerized pipeline with CLI, the (mostly python) code, PyTorch dataloaders for the generated data, the generated starter dataset, download scripts and other utilities are all available ${\color{Magenta}through}\;{\color{Magenta}our}\;{\color{Magenta}project}\;{\color{Magenta}website}$.
Immersive virtual environments for occupant comfort and adaptive behavior research – A comprehensive review of tools and applications

Immersive virtual reality (IVR) is a promising technology increasingly applied to various research fields, including building science and occupant comfort research. Using immersive virtual environments (IVEs), researchers explore important research questions related to how occupants perceive and interact with their indoor environment in response to different environmental stimuli (e.g., visual, thermal, and acoustic). Despite an increasing number of applications, the literature lacks review efforts of IVE-based comfort research, leaving the current state of such efforts, their shortcomings, and prospects unclear. The current work aims to address this stated gap by presenting a comprehensive review of scholarly articles using IVEs to study occupant comfort and related adaptive behaviors in buildings. The document starts with a review of available IVR technologies and their associated fundamental concepts (i.e., immersion, ecological validity, presence, and user experience), followed by a review of studies that employed IVEs to assess comfort or adaptive behaviors across single or multiple environmental domains. A synthesis is then presented to reflect on the current state-of-the-art, its gaps, and the way forward. Results show that despite significant advancements in IVR technology, IVE-based multi-domain comfort and behavior experiments remain very limited (e.g., considering cross-effects between thermal, visual, and acoustic domains). Furthermore, the use of multi-user IVE is particularly scarce and should be further leveraged in the future to understand how groups of individuals (e.g., co-workers or classmates) collectively respond to environmental stimuli, interact with various building systems, and drive building performance.
MFNet: Multi-filter Directive Network for Weakly Supervised Salient Object Detection

Weakly supervised salient object detection (WSOD) targets to train a CNNs-based saliency network using only low-cost annotations. Existing WSOD methods take various techniques to pursue single "high-quality" pseudo label from low-cost annotations and then develop their saliency networks. Though these methods have achieved good performance, the generated single label is inevitably affected by adopted refinement algorithms and shows prejudiced characteristics which further influence the saliency networks. In this work, we introduce a new multiple-pseudo-label framework to integrate more comprehensive and accurate saliency cues from multiple labels, avoiding the aforementioned problem. Specifically, we propose a multi-filter directive network (MFNet) including a saliency network as well as multiple directive filters. The directive filter (DF) is designed to extract and filter more accurate saliency cues from the noisy pseudo labels. The multiple accurate cues from multiple DFs are then simultaneously propagated to the saliency network with a multi-guidance loss. Extensive experiments on five datasets over four metrics demonstrate that our method outperforms all the existing con-generic methods. Moreover, it is also worth noting that our framework is flexible enough to apply to existing methods and improve their performance. The code and results of our method are available at https://github.com/OIPLab-DUT/MFNet.
Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching

Federated learning (FL) has emerged with increasing popularity to collaborate distributed medical institutions for training deep networks. However, despite existing FL algorithms only allow the supervised training setting, most hospitals in realistic usually cannot afford the intricate data labeling due to absence of budget or expertise. This paper studies a practical yet challenging FL problem, named Federated Semi-supervised Learning (FSSL), which aims to learn a federated model by jointly utilizing the data from both labeled and unlabeled clients (i.e., hospitals). We present a novel approach for this problem, which improves over traditional consistency regularization mechanism with a new inter-client relation matching scheme. The proposed learning scheme explicitly connects the learning across labeled and unlabeled clients by aligning their extracted disease relationships, thereby mitigating the deficiency of task knowledge at unlabeled clients and promoting discriminative information from unlabeled samples. We validate our method on two large-scale medical image classification datasets. The effectiveness of our method has been demonstrated with the clear improvements over state-of-the-arts as well as the thorough ablation analysis on both tasks (Code will be made available at https://github.com/liuquande/FedIRM).