Reviewerly/exHarmony huggingface dataset:
Reflexive combinatory algebras

Abstract We introduce the notion of reflexivity for combinatory algebras. Reflexivity can be thought of as an equational counterpart of the Meyer–Scott axiom of combinatory models, which indeed allows us to characterize an equationally definable counterpart of combinatory models. This new structure, called strongly reflexive combinatory algebra, admits a finite axiomatization with seven closed equations, and the structure is shown to be exactly the retract of combinatory models. Lambda algebras can be characterized as strongly reflexive combinatory algebras that are stable. Moreover, there is a canonical construction of a lambda algebra from a strongly reflexive combinatory algebra. The resulting axiomatization of lambda algebras by the seven axioms for strong reflexivity together with those for stability is shown to correspond to the axiomatization of lambda algebras due to Selinger (2002, J. Funct. Program., 12, 549–566).
Guided Interactive Video Object Segmentation Using Reliability-Based  Attention Maps

We propose a novel guided interactive segmentation (GIS) algorithm for video objects to improve the segmentation accuracy and reduce the interaction time. First, we design the reliability-based attention module to analyze the reliability of multiple annotated frames. Second, we develop the intersection-aware propagation module to propagate segmentation results to neighboring frames. Third, we introduce the GIS mechanism for a user to select unsatisfactory frames quickly with less effort. Experimental results demonstrate that the proposed algorithm provides more accurate segmentation results at a faster speed than conventional algorithms. Codes are available at https://github.com/yuk6heo/GIS-RAmap.
Review of end-to-end speech synthesis technology based on deep learning

As an indispensable part of modern human-computer interaction system, speech synthesis technology helps users get the output of intelligent machine more easily and intuitively, thus has attracted more and more attention. Due to the limitations of high complexity and low efficiency of traditional speech synthesis technology, the current research focus is the deep learning-based end-to-end speech synthesis technology, which has more powerful modeling ability and a simpler pipeline. It mainly consists of three modules: text front-end, acoustic model, and vocoder. This paper reviews the research status of these three parts, and classifies and compares various methods according to their emphasis. Moreover, this paper also summarizes the open-source speech corpus of English, Chinese and other languages that can be used for speech synthesis tasks, and introduces some commonly used subjective and objective speech quality evaluation method. Finally, some attractive future research directions are pointed out.
Learning to Cascade: Confidence Calibration for Improving the Accuracy  and Computational Cost of Cascade Inference Systems

Recently, deep neural networks have become to be used in a variety of applications. While the accuracy of deep neural networks is increasing, the confidence score, which indicates the reliability of the prediction results, is becoming more important. Deep neural networks are seen as highly accurate but known to be overconfident, making it important to calibrate the confidence score. Many studies have been conducted on confidence calibration. They calibrate the confidence score of the model to match its accuracy, but it is not clear whether these confidence scores can improve the performance of systems that use confidence scores. This paper focuses on cascade inference systems, one kind of systems using confidence scores, and discusses the desired confidence score to improve system performance in terms of inference accuracy and computational cost. Based on the discussion, we propose a new confidence calibration method, Learning to Cascade. Learning to Cascade is a simple but novel method that optimizes the loss term for confidence calibration simultaneously with the original loss term. Experiments are conducted using two datasets, CIFAR-100 and ImageNet, in two system settings, and show that naive application of existing calibration methods to cascade inference systems sometimes performs worse. However, Learning to Cascade always achieves a better trade-off between inference accuracy and computational cost. The simplicity of Learning to Cascade allows it to be easily applied to improve the performance of existing systems.
Spatial Orientation in Cardiac Ultrasound Images Using Mixed Reality: Design and Evaluation

Spatial orientation is an important skill in structural cardiac imaging. Until recently, 3D cardiac ultrasound has been visualized on a flat screen by using volume rendering. Mixed reality devices enhance depth perception, spatial awareness, interaction, and integration in the physical world, which can prove advantageous with 3D cardiac ultrasound images. In this work, we describe the design of a system for rendering 4D (3D + time) cardiac ultrasound data as virtual objects and evaluate it for ease of spatial orientation by comparing it with a standard clinical viewing platform in a user study. The user study required eight participants to do timed tasks and rate their experience. The results showed that virtual objects in mixed reality provided easier spatial orientation and morphological understanding despite lower perceived image quality. Participants familiar with mixed reality were quicker to orient in the tasks. This suggests that familiarity with the environment plays an important role, and with improved image quality and increased use, mixed reality applications may perform better than conventional 3D echocardiography viewing systems.
Cognitive IoT Vision System Using Weighted Guided Harris Corner Feature Detector for Visually Impaired People

India has an estimated 12 million visually impaired people and is home to the world’s largest number in any country. Smart walking stick devices use various technologies including machine vision and different sensors for improving the safe movement of visually impaired persons. In machine vision, accurately recognizing an object that is near to them is still a challenging task. This paper provides a system to enable safe navigation and guidance for visually impaired people by implementing an object recognition module in the smart walking stick that uses a local feature extraction method to recognize an object under different image transformations. To provide stability and robustness, the Weighted Guided Harris Corner Feature Detector (WGHCFD) method is proposed to extract feature points from the image. WGHCFD discriminates image features competently and is suitable for different real-world conditions. The WGHCFD method evaluates the most popular Oxford benchmark datasets, and it achieves greater repeatability and matching score than existing feature detectors. In addition, the proposed WGHCFD method is tested with a smart stick and achieves 99.8% recognition rate under different transformation conditions for the safe navigation of visually impaired people.
Multi-View Human Tracking and 3D Localization in Retail

In recent years, retail stores have seen traction in bringing online shopping experience to offline stores via autonomous checkouts. Autonomous checkouts is a computer vision-based technology that needs to understand three human elements within the store: who, where, and doing what. This paper addresses two of the three elements: who and where. It presents an approach to track and localize humans in a multi-view camera system. Traditional methods have limitations as they: (1) fail to overcome substantial occlusion of humans; (2) suffer a lengthy processing time; (3) require a planar homography constraint between camera frames; (4) suffer swapping of labels assigned to a human. The proposed method in this paper handles all the aforementioned limitations. The key idea is to use a hierarchical association model for tracking, which uses each human's clothing features, human pose orientation, and relative depth of joints, and runs at over 23fps.
Multi-Modal Rigid Image Registration and Segmentation Using Multi-Stage Forward Path Regenerative Genetic Algorithm

Medical image diagnosis and delineation of lesions in the human brain require information to combine from different imaging sensors. Image registration is considered to be an essential pre-processing technique of aligning images of different modalities. The brain is a naturally bilateral symmetrical organ, where the left half lobe resembles the right half lobe around the symmetrical axis. The identified symmetry axis in one MRI image can identify symmetry axes in multi-modal registered MRI images instantly. MRI sensors may induce different levels of noise and Intensity Non-Uniformity (INU) in images. These image degradations may cause difficulty in finding true transformation parameters for an optimization technique. We will be investigating the new variant of evolution strategy of genetic algorithm as an optimization technique that performs well even for the high level of noise and INU, compared to Nesterov, Limited-memory Broyden–Fletcher–Goldfarb–Shanno algorithm (LBFGS), Simulated Annealing (SA), and Single-Stage Genetic Algorithm (SSGA). The proposed new multi-modal image registration technique based on a genetic algorithm with increasing precision levels and decreasing search spaces in successive stages is called the Multi-Stage Forward Path Regenerative Genetic Algorithm (MFRGA). Our proposed algorithm is better in terms of overall registration error as compared to the standard genetic algorithm. MFRGA results in a mean registration error of 0.492 in case of the same level of noise (1–9)% and INU (0–40)% in both reference and template image, and 0.317 in case of a noise-free template and reference with noise levels (1–9)% and INU (0–40)%. Accurate registration results in good segmentation, and we apply registration transformations to segment normal brain structures for evaluating registration accuracy. The brain segmentation via registration with our proposed algorithm is better even in cases of high levels of noise and INU as compared to GA and LBFGS. The mean dice similarity coefficient of brain structures CSF, GM, and WM is 0.701, 0.792, and 0.913, respectively.
Research on three-dimensional counting algorithm of high-precision suspended cells based on deep learning

The research on the three-dimensional counting algorithm of suspension cells is very important for the online counting of cells. Suspension cells are usually observed and counted by taking out cell samples under the microscope. This method cannot realize the online counting of cells. At the same time, there will be weak movement of suspended cells in three-dimensional space, making space counting more difficult. Aiming at the above difficulties, a three-dimensional counting algorithm of suspended cells based on deep learning is proposed. The Faster regions with convolutional neural network (R-CNN) framework is used to identify the target cells, the feature pyramid structure is introduced to reduce the feature loss in the fusion process, and the soft non-maximum suppression algorithm is used to remove the bounding box to improve the detection effect. To reduce the manual labeling workload, an automatic labeling method is proposed. The improved Faster RCNN framework achieves the F1-score of more than 98% after identifying cells, and our method is more accurate than other advanced methods. In terms of three-dimensional counting algorithms, the cells are spatially tracked. Cell matching and correction are carried out by extracting multiple feature information of cells, and adhesion cells are extracted separately for priority processing. The algorithm carried out a tracking experiment on the image sequences of five groups of suspended cells, and the final tracking accuracy of the experiment reached more than 95%. The tracking accuracy is better than other advanced algorithms. Experimental results show that the algorithm achieves high-precision three-dimensional tracking and counting of suspended cells.
Deep Bayesian Active-Learning-to-Rank for Endoscopic Image Data

Automatic image-based disease severity estimation generally uses discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult due to the images with ambiguous severity. An easier alternative is to use relative annotation, which compares the severity level between image pairs. By using a learning-to-rank framework with relative annotation, we can train a neural network that estimates rank scores that are relative to severity levels. However, the relative annotation for all possible pairs is prohibitive, and therefore, appropriate sample pair selection is mandatory. This paper proposes a deep Bayesian active-learning-to-rank, which trains a Bayesian convolutional neural network while automatically selecting appropriate pairs for relative annotation. We confirmed the efficiency of the proposed method through experiments on endoscopic images of ulcerative colitis. In addition, we confirmed that our method is useful even with the severe class imbalance because of its ability to select samples from minor classes automatically.
How Effective is Adversarial Training of CNNs in Medical Image Analysis?

Adversarial attacks are carefully crafted inputs that can deceive machine learning models into giving wrong results with seemingly high confidence. One approach that is commonly used in the image analysis literature to defend against such attacks is the introduction of adversarial images during training time, i.e. adversarial training. However, the effectiveness of adversarial training remains unclear in the healthcare domain, where the use of complex medical scans is crucial for a wide range of clinical workflows. In this paper, we carried out an empirical investigation into the effectiveness of adversarial training as a defence technique in the context of medical images. We demonstrated that adversarial training is, in principle, a transferable defence on medical imaging data, and that it can potentially be used on attacks previously unseen by the model. We also empirically showed that the strength of the attack, determined by the parameter $$\epsilon $$ , and the percentage of adversarial images included during training, have key influence over the level of success of the defence. Our analysis was carried out using 58,954 images from the publicly available MedNIST benchmarking dataset.
Leveraging Uncertainty in Deep Learning for Pancreatic Adenocarcinoma Grading

Pancreatic cancers have one of the worst prognoses compared to other cancers, as they are diagnosed when cancer has progressed towards its latter stages. The current manual histological grading for diagnosing pancreatic adenocarcinomas is time-consuming and often results in misdiagnosis. In digital pathology, AI-based cancer grading must be extremely accurate in prediction and uncertainty quantification to improve reliability and explainability and are essential for gaining clinicians’ trust in the technology. We present Bayesian Convolutional Neural Networks for automated pancreatic cancer grading from MGG and H&E stained images to estimate uncertainty in model prediction. We show that the estimated uncertainty correlates with prediction error. Specifically, it is useful in setting the acceptance threshold using a metric that weighs classification accuracy-reject trade-off and misclassification cost controlled by hyperparameters and can be employed in clinical settings.
Thigh and Calf Muscles Segmentation Using Ensemble of Patch-Based Deep Convolutional Neural Network on Whole-Body Water-Fat MRI

Lower limb muscles provide various significant physiological functions, which are the key muscles to support daily activities. Segmentation of these muscles offers quantitative analysis for several clinical applications (e.g., understanding of musculoskeletal diseases). In recent studies on muscle tissues, water-fat quantitative magnetic resonance imaging (MRI) is frequently used as the main image modality due to its capability in distinguishing muscle and fat from their surrounding tissues. However, manual muscle segmentation in MRI is time-consuming and often requires professional knowledge. In this paper, an ensemble framework of combining two patch-based binary-class deep convolutional neural networks with the same architecture is proposed to achieve thigh and calf muscle segmentation from whole-body water-fat (mDIXON) MRIs. We compared our model to a state-of-the-art multi-class 3D U-Net model using a 5-fold cross-validation. A Dice coefficient of 0.9042 was achieved by our method, which was significantly more accurate than the multi-class 3D U-Net method. Additionally, the model deployment costs approximately 13.8 s per case.
Rotation-Equivariant Semantic Instance Segmentation on Biomedical Images

Advances in image segmentation techniques, brought by convolutional neural network (CNN) architectures like U-Net, show promise for tasks such as automated cancer screening. Recently, these methods have been extended to detect different instances of the same class, which could be used to, for example, characterize individual cells in whole-slide images. Still, the amount of data needed and the number of parameters in the network are substantial. To alleviate these problems, we modify a method of semantic instance segmentation to also enforce equivariance to the p4 symmetry group of 90-degree rotations and translations. We perform four experiments on a synthetic dataset of scattered sticks and a subset of the Kaggle 2018 Data Science Bowl, the BBBC038 dataset, consisting of segmented nuclei images. Results indicate that the rotation-equivariant architecture yields similar accuracy as a baseline architecture. Furthermore, we observe that the rotation-equivariant architecture converges faster than the baseline. This is a promising step towards reducing the training time during development of methods based on deep learning.
From Astronomy to Histology: Adapting the FellWalker Algorithm to Deep Nuclear Instance Segmentation

Accurate cell nuclei segmentation is necessary for subsequent histopathology image analysis, including tumour classification, grading and prognosis. Manually identifying cell nuclei is both difficult and time-consuming, with cell nuclei exhibiting dramatic differences in morphology and staining characteristics. Recently, significant advancements in automatic cell nuclei segmentation have been achieved using deep learning, with methods particularly successful in identifying cell nuclei from background tissue. However, delineating individual cell nuclei remains challenging, with often unclear boundaries between neighbouring nuclei. In this paper, we incorporate the FellWalker algorithm, originally developed for analysing molecular clouds, into a deep learning-based pipeline to perform instance cell nuclei segmentation. We evaluate our proposed method on the Lizard dataset, the largest publicly available nuclear segmentation dataset in digital pathology, and compare it against popular methods such as U-Net with Watershed and Mask R-CNN. Our proposed method consistently outperforms the other methods across dataset sizes, achieving an object Dice of 0.7876, F1 score of 0.8245 and Aggregated Jaccard Index of 0.6526. The flexible nature of our pipeline incorporating the FellWalker algorithm has the potential for broader application in biomedical image instance segmentation tasks.
EHR Data: Enabling Clinical Surveillance and Alerting

This chapter describes clinical needs, foundation for clinical alerting and surveillance, It started with historical perspective and outlining public health needs. Emergency services provide rapid response for deteriorating patients that use number of assessments tools. Emerging Clinical Decision Support Systems moved to more advanced systems that required better evaluation and adoption. This chapter provides foundation for understanding clinical impact. At the end of the chapter, there is brief case study of Clinical Control Tower.
A Novel Framework for Coarse-Grained Semantic Segmentation of Whole-Slide Images

Semantic segmentation of multi-gigapixel whole-slide images (WSI) is fundamental to computational pathology, as segmentation of different tissue types and layers is a prerequisite for several downstream histology image analysis, such as morphometric analysis, cancer grading, and survival. Both patch-based classification and pixel-wise segmentation have been used for these tasks, where patch-based classification outputs only one label per patch while pixel-wise segmentation is more accurate and precise but it requires a large number of pixel-wise precise annotated ground truth. In this paper, we propose coarse segmentation as a new middle ground to both techniques for leveraging more context without requiring pixel-level annotations. Our proposed coarse segmentation network is a convolutional neural network (CNN) with skip connections but does not contain any decoder and utilizes sparsely annotated images during training. It takes an input patch of size $$M \times N$$ and outputs a dense prediction map of size $$m \times n$$ , which is coarser than pixel-wise segmentation methods but denser than patch-based classification methods. We compare our proposed method with its counterparts and demonstrate its superior performance for both pixel-based segmentation and patch-based classification tasks. In addition, we also compared the impact on performance of coarse-grained and pixel-wise semantic segmentation in downstream analysis tasks and showed coarse-grained semantic segmentation has no/marginal impact on the final results.
Polyp2Seg: Improved Polyp Segmentation with Vision Transformer

Colorectal cancer (CRC) is the third most common type of cancer worldwide. It can be prevented by screening the colon and detecting polyps which might become malign. Therefore, an accurate detection/segmentation of polyps in colonoscopy images is crucial for CRC prevention. In this paper, we propose a novel transformer-based architecture for polyp image segmentation named Polyp2Seg. The model adopts a transformer architecture as its encoder to extract multi-hierarchical features. Additionally, a novel Feature Aggregation Module (FAM) merges progressively the multi-level features from the encoder to better localise polyps by adding semantic information. Next, a Multi-Context Attention Module (MCAM) removes noise and other artifacts, while incorporating a multi-scale attention mechanism to further improve polyp detections. Quantitative and qualitative experiments on five challenging datasets and over 5 different SOTAs demonstrate that our method significantly improves the segmentation accuracy of Polyps under different evaluation metrics. Our model achieves a new state-of-the-art over most of the datasets.
Weakly Supervised Captioning of Ultrasound Images

Medical image captioning models generate text to describe the semantic contents of an image, aiding the non-experts in understanding and interpretation. We propose a weakly-supervised approach to improve the performance of image captioning models on small image-text datasets by leveraging a large anatomically-labelled image classification dataset. Our method generates pseudo-captions (weak labels) for caption-less but anatomically-labelled (class-labelled) images using an encoder-decoder sequence-to-sequence model. The augmented dataset is used to train an image-captioning model in a weakly supervised learning manner. For fetal ultrasound, we demonstrate that the proposed augmentation approach outperforms the baseline on semantics and syntax-based metrics, with nearly twice as much improvement in value on BLEU-1 and ROUGE-L. Moreover, we observe that superior models are trained with the proposed data augmentation, when compared with the existing regularization techniques. This work allows seamless automatic annotation of images that lack human-prepared descriptive captions for training image-captioning models. Using pseudo-captions in the training data is particularly useful for medical image captioning when significant time and effort of medical experts is required to obtain real image captions.