Reviewerly/exHarmony huggingface dataset:
A Survey of Natural Design for Interaction

The term “Natural Design” has various meanings and applications within and beyond the human-computer interaction community. Yet, there is no consensus on whether it is a relevant design approach or only a descriptive term without profound meaning. We investigated the current understanding and design potential of “Natural Design” for interaction in a systematic literature review. By analyzing and rating 113 papers, we identified 47 relevant papers that applied Natural Design in different contexts. The understanding of the approach changes from nature-related inspirations to context-dependent naturalness based on increasing familiarity or expectations. We present a structured overview of these relevant papers, contribute a systematic Natural Design model for interaction and add 20 implications for applying Natural Design to natural user interfaces, natural interaction, or computation. We identified “Natural Design” as a relevant design approach to create intuitive and embedded interfaces that can profit from related concepts outside human-computer interaction.
Age, Gender, and Gesture Classification Using Open-Source Computer Vision

In every country, there are a plethora of laws whose very foundation stands on the age of the concerned. Similarly, successful gender recognition is essential and critical for many applications in the commercial domains, like human–computer interactions: such as computer-aided physiological or psychological analysis of a human. In this work, a face and gesture detection and verification system are proposed that classifies them on the basis of gender while providing the most probable age range of the concerned face and also detecting the gesture of the hands using convolutional neural network architecture. The principal idea behind the system is to compare the image with the reference images stored as templates in the database and to determine the age and gender.
CAMDNN: Content-Aware Mapping of a Network of Deep Neural Networks on Edge MPSoCs

Machine Learning (ML) workloads are increasingly deployed at the edge. Enabling efficient inference execution while considering model and system heterogeneity remains challenging, especially for ML tasks built with a network of DNNs. The challenge is to maximize the utilization of all available resources on the multiprocessor system on a chip (MPSoC) at the same time. This becomes even more complicated because the optimal mapping for the network of DNNs can vary with input batch sizes and scene complexity. In this paper, a holistic hierarchical scheduling framework is presented to optimize the execution time for a network of DNN models on an edge MPSoC at runtime, considering varying input characteristics. The framework consists of a local and a global scheduler. The local scheduler maps individual DNNs in the inference pipeline to the best-performing hardware unit while the global scheduler customizes an Integer Linear Programming (ILP) solution to instantiate DNN remapping. To minimize scheduler runtime overhead, an imitation learning (IL) based scheduler is used that approximates the ILP solutions. The proposed scheduling framework (CAMDNN) was implemented on a Qualcomm Robotic RB5 platform. CAMDNN resulted in lower execution time of up to 32% than HEFT, and by factors of 6.67X, 5.6X and 2.17X than the CPU-only, GPU-only and Central Queue schedulers.
When the Closest Targets Make the Difference: An Analysis of Data Association Errors

Multi-object data fusion – combining measurements or estimates of more than one truth object from more than one observer – requires a first “data association” step of deciding deciding which data have common truth objects. Here we focus on the case of two observers only, with the data association engine powered by a polynomially-complex list-matching algorithm such as of Jonker-Volgenant-Castanon (JVC), auction or Munkres. The paper's purpose is to develop an approximation for the probability of assignment error: How often does the data association engine tell the fuser to combine data from truth objects that do not go together? We assume data with Gaussian errors and a Poisson field of truth objects, and we focus on the low-noise case where errors are infrequent and fusion makes sense. In this article, for isotropic, independent identically distributed errors, a single scalar parameter representative of the scene complexity is identified and, exploiting that, a reasonably simple approximate expression for the association error is derived.
MPTC-FPN: A Multilayer Progressive FPN With Transformer-CNN Based Encoder for Salient Object Detection

Due to the development of Convolutional Neural Networks (CNN), significant progress has been made in Salient Object Detection (SOD). However, methods based on CNN are difficult to achieve good results in learning global context information. Recently, with the rapid development of vision transformer, it provides a new perspective for the performance improvement of salient object detection.Benefiting from the powerful capability of global modeling, transformer can supplement rich global contextual information. For lacking the ability to learn local details, it is suboptimal to only adopt transformer as encoder. Therefore, how to skillfully combine local details and global context information is crucial. We conbine CNN and transformer to propose a Multilayer Progressive FPN with Transformer-CNN Based Encoder For Salient Object Detection (MPTC-FPN). Similar to most of the previous methods, we adopt the FPN network as the basic structure. But the difference from previous methods is that we have six initial features before feature fusion, instead of the traditional four or five. We use a low-level feature generation module (LFGM) to generate a lower-level feature to supplement local details. In addition, we also propose a module to reduce the difference between features (DRM), making the features more conducive to fusion. On the basis of FPN, we add a large number of feature fusion nodes, which makes the process of feature fusion smoother. Moreover, we adjust the supervision strategy, use multiple supervision points, and adopt an appropriate weight distribution strategy among the multiple supervision points. A series of comprehensive experimental results demonstrates that our proposed method outperforms previous state-of-the-art methods on five datasets.
Deep is a Luxury We Don’t Have

Medical images come in high resolutions. A high resolution is vital for finding malignant tissues at an early stage. Yet, this resolution presents a challenge in terms of modeling long range dependencies. Shallow transformers eliminate this problem, but they suffer from quadratic complexity. In this paper, we tackle this complexity by leveraging a linear self-attention approximation. Through this approximation, we propose an efficient vision model called HCT that stands for High resolution Convolutional Transformer. HCT brings transformers' merits to high resolution images at a significantly lower cost. We evaluate HCT using a high resolution mammography dataset. HCT is significantly superior to its CNN counterpart. Furthermore, we demonstrate HCT's fitness for medical images by evaluating its effective receptive field. Code available at https://bit.ly/3ykBhhf .
MaNi: Maximizing Mutual Information for Nuclei Cross-Domain Unsupervised Segmentation

In this work, we propose a mutual information (MI) based unsupervised domain adaptation (UDA) method for the cross-domain nuclei segmentation. Nuclei vary substantially in structure and appearances across different cancer types, leading to a drop in performance of deep learning models when trained on one cancer type and tested on another. This domain shift becomes even more critical as accurate segmentation and quantification of nuclei is an essential histopathology task for the diagnosis/prognosis of patients and annotating nuclei at the pixel level for new cancer types demands extensive effort by medical experts. To address this problem, we maximize the MI between labeled source cancer type data and unlabeled target cancer type data for transferring nuclei segmentation knowledge across domains. We use the Jensen-Shanon divergence bound, requiring only one negative pair per positive pair for MI maximization. We evaluate our set-up for multiple modeling frameworks and on different datasets comprising of over 20 cancer-type domain shifts and demonstrate competitive performance. All the recently proposed approaches consist of multiple components for improving the domain adaptation, whereas our proposed module is light and can be easily incorporated into other methods (Implementation: https://github.com/YashSharma/MaNi ).
Heterogeneity Shifts the Storage-Computation Tradeoff in Secure Multi-Cloud Systems

This paper considers the design of heterogeneous multi-cloud systems for big data storage and computing in the presence of cloud collusion and failures. A fundamental concept of such a system is the secrecy capacity, which represents the maximum amount of information that can be stored for each unit of storage space under the requirements of secure distributed computing. A capacity-achieving code is designed for matrix multiplication, a computing subroutine widely used in machine learning applications. The code allows fast parallel decoding and unequal data allocation in the clouds. Such a flexibility leads naturally to the idea of optimizing data allocation to minimize the computing time. Given any feasible storage budget, the optimal solution is derived, characterizing explicitly the fundamental tradeoff between storage and computing. Furthermore, it is shown via majorization theory that the whole tradeoff curve improves if the cloud computing rates are more even. Experiments on Amazon EC2 clusters are conducted, corroborating our theoretical observations and the negligibility of decoding overhead.
Efficient SPARQL Queries Generator for Question Answering Systems

Much like traditional database querying, the question answering process in a Question Answering (QA) system involves converting a user’s question input into query grammar, querying the knowledge base through the query grammar, and finally returning the query result (i.e., the answer) to the user. The accuracy of query grammar generation is therefore important in determining whether a Question Answering system can produce a correct answer. Generally speaking, incorrect query grammar will never find the right answer. SPARQL is the most frequently used query language in question answering systems. In the past, SPARQL was generated based on graph structures, such as dependency trees, syntax trees and so on. However, the query cost of generating SPARQL is high, which creates long processing times to answer questions. To reduce the query cost, this work proposes a low-cost SPARQL generator named Light-QAWizard, which integrates multi-label classification into a recurrent neural network (RNN), builds a template classifier, and generates corresponding query grammars based on the results of template classifier. Light-QAWizard reduces query frequency to DBpedia by aggregating multiple outputs into a single output using multi-label classification. In the experimental results, Light-QAWizard’s performance on Precision, Recall and F-measure metrics were evaluated on the QALD-7, QALD8 and QALD-9 datasets. Not only did Light-QAWizard outperform all other models, but it also had a lower query cost that was nearly half that of QAWizard.
Language-Based Image Manipulation Built on Language-Guided Ranking

Text-based image manipulation is a popular subject and has many applications. However, it is a challenging task because there is no ground-truth edited dataset and textual descriptions have abstractive and ambiguous properties. To alleviate the difficult issues, we propose a manipulation framework consisting of the proposal attentional GANs, language-related semantic mask, and language-guided ranker. Specially, we construct an editing proposal generator to generate the suitable edited proposals with and without semantic conditions, which supports the reorganization of sub-generators to output proposals in various aspects as many as possible. To distinguish the text-relevant and the text-irrelevant regions, we introduce a language-related semantic mask based on the source image and target caption. Then, we exploit a language-guided ranker to retrieve the best edited result from the edited proposals through using the multi-modal similarity and the language-related semantic mask. Extensive experiments on widely-used datasets demonstrate that our model could manipulate images interactively and improve the editing quality effectively.
Sample Hardness Based Gradient Loss for Long-Tailed Cervical Cell Detection

Due to the difficulty of cancer samples collection and annotation, cervical cancer datasets usually exhibit a long-tailed data distribution. When training a detector to detect the cancer cells in a WSI (Whole Slice Image) image captured from the TCT (Thinprep Cytology Test) specimen, head categories (e.g. normal cells and inflammatory cells) typically have a much larger number of samples than tail categories (e.g. cancer cells). Most existing state-of-the-art long-tailed learning methods in object detection focus on category distribution statistics to solve the problem in the long-tailed scenario, without considering the "hardness" of each sample. To address this problem, in this work we propose a Grad-Libra Loss that leverages the gradients to dynamically calibrate the degree of hardness of each sample for different categories, and re-balance the gradients of positive and negative samples. Our loss can thus help the detector to put more emphasis on those hard samples in both head and tail categories. Extensive experiments on a long-tailed TCT WSI image dataset show that the mainstream detectors, e.g. RepPoints, FCOS, ATSS, YOLOF, etc. trained using our proposed Gradient-Libra Loss, achieved much higher (7.8%) mAP than that trained using cross-entropy classification loss.
Semi-supervised PR Virtual Staining for Breast Histopathological Images

Progesterone receptor (PR) plays a vital role in diagnosing and treating breast cancer, but PR staining is costly and time-consuming, seriously hindering its application in clinical practice. The recent rapid development of deep learning technology provides an opportunity to address this problem by virtual staining. However, supervised methods acquire pixel-level paired H &E and PR images, which almost cannot be implemented clinically. In addition, unsupervised methods lack effective constraint information, and the staining results are not reliable sometimes. In this paper, we propose a semi-supervised PR virtual staining method without any pathologist annotation. Firstly, we register the consecutive slides and obtain the patch-level labels of H &E images from the registered consecutive PR images. Furthermore, by designing a Pos/Neg classifier and corresponding constraints, the output images maintain the Pos/Neg consistency with the input images, enabling the output images to be more accurate. Experimental results show that our method can effectively generate PR images from H &E images and maintain structural and pathological consistency with the reference. Compared with existing methods, our approach achieves the best performance.
Geometric Constraints for Self-supervised Monocular Depth Estimation on Laparoscopic Images with Dual-task Consistency

Depth values are essential information to automate surgical robots and achieve Augmented Reality technology for minimally invasive surgery. Although depth-pose self-supervised monocular depth estimation performs impressively for autonomous driving scenarios, it is more challenging to predict accurate depth values for laparoscopic images due to the following two aspects: (i) the laparoscope's motions contain many rotations, leading to pose estimation difficulties for the depth-pose learning strategy; (ii) the smooth surface reduces photometric error even if the matching pixels are inaccurate between adjacent frames. This paper proposes a novel self-supervised monocular depth estimation for laparoscopic images with geometric constraints. We predict the scene coordinates as an auxiliary task and construct dual-task consistency between the predicted depth maps and scene coordinates under a unified camera coordinate system to achieve pixel-level geometric constraints. We extend the pose estimation into a Siamese process to provide stronger and more balanced geometric constraints in a depth-pose learning strategy by leveraging the order of the adjacent frames in a video sequence. We also design a weight mask for depth estimation based on our consistency to alleviate the interference from predictions with low confidence. The experimental results showed that the proposed method outperformed the baseline on depth and pose estimation. Our code is available at https://github.com/MoriLabNU/GCDepthL .
Evaluating Attribution Methods for Explainable NLP with Transformers

This paper describes the experimental evaluation of several attribution methods on two NLP tasks: Sentiment analysis and multi-label document classification. Our motivation is to find the best method to use with Transformers to interpret model decisions. For this purpose, we introduce two new evaluation datasets. The first one is derived from Stanford Sentiment Treebank, where the sentiment of individual words is annotated along with the sentiment of the whole sentence. The second dataset comes from Czech Text Document Corpus, where we added keyword information assigned to each category. The keywords were manually assigned to each document and automatically propagated to categories via PMI. We evaluate each attribution method on several models of different sizes. The evaluation results are reasonably consistent across all models and both datasets. It indicates that both datasets with proposed evaluation metrics are suitable for interpretability evaluation. We show how the attribution methods behave concerning model size and task. We also consider practical applications – we show that while some methods perform well, they can be replaced with slightly worse-performing methods requiring significantly less time to compute.
Clinical-Realistic Annotation for Histopathology Images with Probabilistic Semi-supervision: A Worst-Case Study

Acquiring pixel-level annotation has been a major challenge for machine learning methods in medical image analysis. Such difficulty mainly comes from two sources: localization requiring high expertise, and delineation requiring tedious and time-consuming work. Existing methods of easing the annotation effort mostly focus on the latter one, the extreme of which is replacing the delineation with a single label for all cases. We postulate that under a clinical-realistic setting, such methods alone may not always be effective in reducing the annotation requirements from conventional classification/detection algorithms, because the major difficulty can come from localization, which is often neglected but can be critical in medical domain, especially for histopathology images. In this work, we performed a worst-case scenario study to identify the information loss from missing detection. To tackle the challenge, we 1) proposed a different annotation strategy to image data with different levels of disease severity, 2) combined semi- and self-supervised representation learning with probabilistic weakly supervision to make use of the proposed annotations, and 3) illustrated its effectiveness in recovering useful information under the worst-case scenario. As a shift from previous convention, it can potentially save significant time for experts' annotation for AI model development.
Transforming the Interactive Segmentation for Medical Imaging

The goal of this paper is to interactively refine the automatic segmentation on challenging structures that fall behind human performance, either due to the scarcity of available annotations or the difficulty nature of the problem itself, for example, on segmenting cancer or small organs. Specifically, we propose a novel Transformer-based architecture for Interactive Segmentation (TIS), that treats the refinement task as a procedure for grouping pixels with similar features to those clicks given by the end users. Our proposed architecture is composed of Transformer Decoder variants, which naturally fulfills feature comparison with the attention mechanisms. In contrast to existing approaches, our proposed TIS is not limited to binary segmentations, and allows the user to edit masks for arbitrary number of categories. To validate the proposed approach, we conduct extensive experiments on three challenging datasets and demonstrate superior performance over the existing state-of-the-art methods. The project page is: https://wtliu7.github.io/tis/ .
DeSD: Self-Supervised Learning with Deep Self-Distillation for 3D Medical Image Segmentation

Self-supervised learning (SSL), enabling advanced performance with few annotations, has demonstrated a proven successful in medical image segmentation. Usually, SSL relies on measuring the similarity of features obtained at the deepest layer to attract the features of positive pairs or repulse the features of negative pairs, and then may suffer from the weak supervision at shallow layers. To address this issue, we reformulate SSL in a Deep Self-Distillation (DeSD) manner to improve the representation quality of both shallow and deep layers. Specifically, the DeSD model is composed of an online student network and a momentum teacher network, both being stacked by multiple sub-encoders. The features produced by each sub-encoder in the student network are trained to match the features produced by the teacher network. Such a deep self-distillation supervision is able to improve the representation quality of all sub-encoders, including both shallow ones and deep ones. We pre-train the DeSD model on a large-scale unlabeled dataset and evaluate it on seven downstream segmentation tasks. Our results indicate that the proposed DeSD model achieves superior pre-training performance over existing SSL methods, setting the new state of the art. The code is available at https://github.com/yeerwen/DeSD .
On the Importance of Word Embedding in Automated Harmful Information Detection

Social media have been growing rapidly during past years. They changed different aspects of human life, especially how people communicate and also how people access information. However, along with the important benefits, social media causes a number of significant challenges since they were introduced. Spreading of fake news and hate speech are among the most challenging issues which have attracted a lot of attention by researchers in past years. Different models based on natural language processing are developed to combat these phenomena and stop them in the early stages before mass spreading. Considering the difficulty of the task of automated harmful information detection (i.e., fake news and hate speech detection), every single step of the detection process could have a sensible impact on the performance of models. In this paper, we study the importance of word embedding on the overall performance of deep neural network architecture on the detection of fake news and hate speech on social media. We test various approaches for converting raw input text into vectors, from random weighting to state-of-the-art contextual word embedding models. In addition, to compare different word embedding approaches, we also analyze different strategies to get the vectors from contextual word embedding models (i.e., get the weights from the last layer, against averaging weights of the last layers). Our results show that XLNet embedding outperforms the other embedding approaches on both tasks related to harmful information identification.