Reviewerly/exHarmony huggingface dataset:
A Conditional Flow Variational Autoencoder for Controllable Synthesis of Virtual Populations of Anatomy

The generation of virtual populations (VPs) of anatomy is essential for conducting in silico trials of medical devices. Typically, the generated VP should capture sufficient variability while remaining plausible and should reflect the specific characteristics and demographics of the patients observed in real populations. In several applications, it is desirable to synthesise virtual populations in a controlled manner, where relevant covariates are used to conditionally synthesise virtual populations that fit a specific target population/characteristics. We propose to equip a conditional variational autoencoder (cVAE) with normalising flows to boost the flexibility and complexity of the approximate posterior learnt, leading to enhanced flexibility for controllable synthesis of VPs of anatomical structures. We demonstrate the performance of our conditional flow VAE using a data set of cardiac left ventricles acquired from 2360 patients, with associated demographic information and clinical measurements (used as covariates/conditional information). The results obtained indicate the superiority of the proposed method for conditional synthesis of virtual populations of cardiac left ventricles relative to a cVAE. Conditional synthesis performance was evaluated in terms of generalisation and specificity errors and in terms of the ability to preserve clinically relevant biomarkers in synthesised VPs, that is, the left ventricular blood pool and myocardial volume, relative to the real observed population.
Trust Your Neighbours: Penalty-Based Constraints for Model Calibration

Ensuring reliable confidence scores from deep networks is of pivotal importance in critical decision-making systems, notably in the medical domain. While recent literature on calibrating deep segmentation networks has led to significant progress, their uncertainty is usually modeled by leveraging the information of individual pixels, which disregards the local structure of the object of interest. In particular, only the recent Spatially Varying Label Smoothing (SVLS) approach addresses this issue by softening the pixel label assignments with a discrete spatial Gaussian kernel. In this work, we first present a constrained optimization perspective of SVLS and demonstrate that it enforces an implicit constraint on soft class proportions of surrounding pixels. Furthermore, our analysis shows that SVLS lacks a mechanism to balance the contribution of the constraint with the primary objective, potentially hindering the optimization process. Based on these observations, we propose a principled and simple solution based on equality constraints on the logit values, which enables to control explicitly both the enforced constraint and the weight of the penalty, offering more flexibility. Comprehensive experiments on a variety of well-known segmentation benchmarks demonstrate the superior performance of the proposed approach. The code is available at https://github.com/Bala93/MarginLoss .
Dice Semimetric Losses: Optimizing the Dice Score with Soft Labels

The soft Dice loss (SDL) has taken a pivotal role in numerous automated segmentation pipelines in the medical imaging community. Over the last years, some reasons behind its superior functioning have been uncovered and further optimizations have been explored. However, there is currently no implementation that supports its direct utilization in scenarios involving soft labels. Hence, a synergy between the use of SDL and research leveraging the use of soft labels, also in the context of model calibration, is still missing. In this work, we introduce Dice semimetric losses (DMLs), which (i) are by design identical to SDL in a standard setting with hard labels, but (ii) can be employed in settings with soft labels. Our experiments on the public QUBIQ, LiTS and KiTS benchmarks confirm the potential synergy of DMLs with soft labels (e.g. averaging, label smoothing, and knowledge distillation) over hard labels (e.g. majority voting and random selection). As a result, we obtain superior Dice scores and model calibration, which supports the wider adoption of DMLs in practice. The code is available at https://github.com/zifuwanggg/JDTLosses .
SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation

Recent advancements in large-scale Vision Transformers have made significant strides in improving pre-trained models for medical image segmentation. However, these methods face a notable challenge in acquiring a substantial amount of pre-training data, particularly within the medical field. To address this limitation, we present Masked Multi-view with Swin Transformers (SwinMM), a novel multi-view pipeline for enabling accurate and data-efficient self-supervised medical image analysis. Our strategy harnesses the potential of multi-view information by incorporating two principal components. In the pre-training phase, we deploy a masked multi-view encoder devised to concurrently train masked multi-view observations through a range of diverse proxy tasks. These tasks span image reconstruction, rotation, contrastive learning, and a novel task that employs a mutual learning paradigm. This new task capitalizes on the consistency between predictions from various perspectives, enabling the extraction of hidden multi-view information from 3D medical data. In the fine-tuning stage, a cross-view decoder is developed to aggregate the multi-view information through a cross-attention block. Compared with the previous state-of-the-art self-supervised learning method Swin UNETR, SwinMM demonstrates a notable advantage on several medical image segmentation tasks. It allows for a smooth integration of multi-view information, significantly boosting both the accuracy and data-efficiency of the model. Code and models are available at https://github.com/UCSC-VLAA/SwinMM/ .
MetaStream: Live Volumetric Content Capture, Creation, Delivery, and Rendering in Real Time

While recent work explored streaming volumetric content on-demand, there is little effort on live volumetric video streaming that bears the potential of bringing more exciting applications than its on-demand counterpart. To fill this critical gap, in this paper, we propose MetaStream, which is, to the best of our knowledge, the first practical live volumetric content capture, creation, delivery, and rendering system for immersive applications such as virtual, augmented, and mixed reality. To address the key challenge of the stringent latency requirement for processing and streaming a huge amount of 3D data, MetaStream integrates several innovations into a holistic system, including dynamic camera calibration, edge-assisted object segmentation, cross-camera redundant point removal, and foveated volumetric content rendering. We implement a prototype of MetaStream using commodity devices and extensively evaluate its performance. Our results demonstrate that MetaStream achieves low-latency live volumetric video streaming at close to 30 frames per second on WiFi networks. Compared to state-of-the-art systems, MetaStream reduces end-to-end latency by up to 31.7% while improving visual quality by up to 12.5%.
The Role of Geometry in Convolutional Neural Networks for Medical Imaging

Convolutional neural networks (CNNs) have played an important role in medical imaging—from diagnostics to research to data integration. This has allowed clinicians to plan operations, diagnose patients earlier, and study rare diseases in more detail. However, data quality, quantity, and imbalance all pose challenges for CNN training and accuracy; in addition, training costs can be high when many types of CNNs are needed in a health care system. Topology and geometry provide tools to ameliorate these challenges for CNNs when they are integrated into the CNN architecture, particularly in the data preprocessing steps or convolution layers. This paper reviews the current integration of geometric tools within CNN architectures to reduce the burden of large training datasets and offset computational costs. This paper also identifies fertile areas for future research into the integration of geometric tools with CNNs.
Semi-supervised generative adversarial networks with spatial coevolution for enhanced image generation and classification

Labeling images for classification can be expensive. Semi-Supervised Learning (SSL) Generative Adversarial Network (GAN) methods train good classifiers with a few labeled images. However, authors generally do not train SSL-GAN generators to produce new high-quality images, but as a component to train the classifier. In this article, we use a coevolutionary algorithm (CoEA) with SSL-GANs to train both the classifier and the image generative model using a few labeled images. A CoEA introduces diversity into the GAN training and mitigates training pathologies. We use a two-dimensional grid of GANs to inject diversity via distributed training that exchanges GAN components between neighboring cells based on performance and population-based hyperparameter tuning. In addition, we identify simple and efficient SSL-GAN architectures. We demonstrate the utility on three separate benchmark datasets, achieving good classification accuracy and high image quality generation while using fewer labeled data exemplars. The image quality and classification accuracy are also competitive with State-of-the-Art methods.
Multimodal transformer network for incomplete image generation and diagnosis of Alzheimer’s disease

Multimodal images such as magnetic resonance imaging (MRI) and positron emission tomography (PET) could provide complementary information about the brain and have been widely investigated for the diagnosis of neurodegenerative disorders such as Alzheimer's disease (AD). However, multimodal brain images are often incomplete in clinical practice. It is still challenging to make use of multimodality for disease diagnosis with missing data. In this paper, we propose a deep learning framework with the multi-level guided generative adversarial network (MLG-GAN) and multimodal transformer (Mul-T) for incomplete image generation and disease classification, respectively. First, MLG-GAN is proposed to generate the missing data, guided by multi-level information from voxels, features, and tasks. In addition to voxel-level supervision and task-level constraint, a feature-level auto-regression branch is proposed to embed the features of target images for an accurate generation. With the complete multimodal images, we propose a Mul-T network for disease diagnosis, which can not only combine the global and local features but also model the latent interactions and correlations from one modality to another with the cross-modal attention mechanism. Comprehensive experiments on three independent datasets (i.e., ADNI-1, ADNI-2, and OASIS-3) show that the proposed method achieves superior performance in the tasks of image generation and disease diagnosis compared to state-of-the-art methods.
Echo from Noise: Synthetic Ultrasound Image Generation Using Diffusion Models for Real Image Segmentation

Abstract We propose a novel pipeline for the generation of synthetic ultrasound images via Denoising Diffusion Probabilistic Models (DDPMs) guided by cardiac semantic label maps. We show that these synthetic images can serve as a viable substitute for real data in the training of deep-learning models for ultrasound image analysis tasks such as cardiac segmentation. To demonstrate the effectiveness of this approach, we generated synthetic 2D echocardiograms and trained a neural network for segmenting the left ventricle and left atrium. The performance of the network trained on exclusively synthetic images was evaluated on an unseen dataset of real images and yielded mean Dice scores of $$88.6 \pm 4.91$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>88.6</mml:mn> <mml:mo>±</mml:mo> <mml:mn>4.91</mml:mn> </mml:mrow> </mml:math> , $$91.9 \pm 4.22$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>91.9</mml:mn> <mml:mo>±</mml:mo> <mml:mn>4.22</mml:mn> </mml:mrow> </mml:math> , $$85.2 \pm 4.83$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>85.2</mml:mn> <mml:mo>±</mml:mo> <mml:mn>4.83</mml:mn> </mml:mrow> </mml:math> % for left ventricular endocardium, epicardium and left atrial segmentation respectively. This represents a relative increase of 9.2, 3.3 and 13.9% in Dice scores compared to the previous state-of-the-art. The proposed pipeline has potential for application to a wide range of other tasks across various medical imaging modalities.
Metric First-Order Temporal Logic with Complex Data Types

Temporal logics are widely used in runtime verification as they enable the creation of declarative and compositional specifications. However, their ability to model complex data is limited. One must resort to complicated encoding schemes to express properties involving basic structures such as lists or trees. To avoid this drawback, we extend metric first-order temporal logic with a minimalistic, yet expressive, functional programming language. The extension features an expressive collection of types including function, record, variant, and inductive types, as well as support for type inference and monitoring. Our monitor implementation directly parses traces in the JSON format, based on the user’s type specification, which avoids a separate pre-processing step. We compare our approach to existing shallow embeddings of temporal properties in general-purpose host languages and to encodings into simple temporal logics. Specifically, our language benefits from a precise semantics and a good support for monitoring-specific static analysis.
Semantic Segmentation of Surgical Hyperspectral Images Under Geometric Domain Shifts

Robust semantic segmentation of intraoperative image data could pave the way for automatic surgical scene understanding and autonomous robotic surgery. Geometric domain shifts, however – although common in real-world open surgeries due to variations in surgical procedures or situs occlusions – remain a topic largely unaddressed in the field. To address this gap in the literature, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation networks in the presence of geometric out-of-distribution (OOD) data, and (2) address generalizability with a dedicated augmentation technique termed 'Organ Transplantation' that we adapted from the general computer vision community. According to a comprehensive validation on six different OOD data sets comprising 600 RGB and yperspectral imaging (HSI) cubes from 33 pigs semantically annotated with 19 classes, we demonstrate a large performance drop of SOA organ segmentation networks applied to geometric OOD data. Surprisingly, this holds true not only for conventional RGB data (drop of Dice similarity coefficient (DSC) by 46 %) but also for HSI data (drop by 45 %), despite the latter's rich information content per pixel. Using our augmentation scheme improves on the SOA DSC by up to 67% (RGB) and 90% (HSI)) and renders performance on par with in-distribution performance on real OOD test data. The simplicity and effectiveness of our augmentation scheme makes it a valuable network-independent tool for addressing geometric domain shifts in semantic scene segmentation of intraoperative data. Our code and pre-trained models are available at https://github.com/IMSY-DKFZ/htc .
The Emergence of Threat Infrastructures: Plan S and Behavioral Change

Defining Threat Infrastructures ‘Threat infrastructures’ are platforms that are established or promised to be established solely or primarily in order to change the behavior of incumbent initiatives through fear. In recent years, such platforms have featured heavily in the scholarly communications landscape and have been driven primarily by funders pushing for open access.
Revolutionizing educational landscapes: A systematic review of Metaverse applications, paradigms and emerging technologies

This paper addresses the existing gap in the literature on Metaverse applications in education by providing a comprehensive analysis of associated paradigms, platforms, hardware, and software, which have not been systematically reviewed yet. Notably, this study is among the first, to the best of our knowledge, to introduce the pioneering concept of Meta-AI in education and propose its potential to revolutionize the educational landscape by synergistically combining generative AI technologies, such as ChatGPT, with Metaverse education. Through this in-depth examination, our research makes a significant contribution to the field, establishing a solid foundation for the development of effective Metaverse teaching applications and guiding future scholarly endeavours.
ChildGAN: Large Scale Synthetic Child Facial Data Using Domain Adaptation in StyleGAN

In this research work, we proposed a novel ChildGAN, a pair of GAN networks for generating synthetic boys and girls facial data derived from StyleGAN2. ChildGAN is built by performing smooth domain transfer using transfer learning. It provides photo-realistic, high-quality data samples. A large-scale dataset is rendered with a variety of smart facial transformations: facial expressions, age progression, eye blink effects, head pose, skin and hair color variations, and variable lighting conditions. The dataset comprises more than 300k distinct data samples. Further, the uniqueness and characteristics of the rendered facial features are validated by running different computer vision application tests which include CNN-based child gender classifier, face localization and facial landmarks detection test, identity similarity evaluation using ArcFace, and lastly running eye detection and eye aspect ratio tests. The results demonstrate that synthetic child facial data of high quality offers an alternative to the cost and complexity of collecting a large-scale dataset from real children. The complete dataset along with the trained model are open-sourced on our GitHub website and GitHub page: https://github.com/MAli-Farooq/ChildGAN.
Hybrid ant colony and intelligent water drop algorithm for route planning of unmanned aerial vehicles

Route planning is a crucial element in unmanned aerial vehicle (UAV) systems, particularly in autonomous UAV technology. In the past decades, various algorithms have been proposed for UAV route planning. However, they still have defects, such as stagnation and slow search rates. In this study, a novel hybrid algorithm which integrates ant colony optimization (ACO) and intelligent water drop (IWD) is proposed for UAV route planning. First, the advantages of the IWD and ACO algorithms are combined in an iterative strategy, to ensure mutual cooperation via exchange of information for route optimization. Initially, the water drops optimize the soil and pheromones within the environment simultaneously to generate good approximate solutions and an initial pheromone distribution for the ant colony. Based thereupon, the ant colony roams the solution space to further optimize the routes, thus combining the exploration potential and exploitability of two types of agents. Additionally, a novel node selection strategy is proposed to guide the agents’ route planning along a reasonable direction. Compared with state-of-the-art algorithms, the convergence accuracy, success rate, and stability of the proposed algorithm exhibited significant improvements of approximately 8.25%, 4.20%, and 66.20%, respectively.
3D Object 360-Degree Motion Detection Using Ultra-Frequency PIR Sensor

This research paper introduces a groundbreaking 3D object 360 motion detection system that employs radiation-based active motion detectors. These sophisticated detectors emit electromagnetic waves and meticulously measure the energy differentials pre and post-reflection to discern motion. The system further harnesses the power of wavelet representation for object recognition amidst crowded scenes, harnessing the prowess of dimensionality reduction and support vector machines for efficient and effective learning. Additionally, the paper explores advanced shape detection techniques, encompassing cutting-edge edge detection and precise geometric parameter estimation. It delves into the efficacy of frame subtraction and background subtraction as highly adaptable motion detection methodologies. The study also sheds light on the merits and demerits of diverse motion detection technologies, encompassing infrared sensors, video motion detection, thermal imaging cameras, and radar systems. Through its comprehensive analysis, this research paves the way for future investigations aimed at enhancing accuracy and performance in 3D object 360 motion detection, thus catering to the evolving needs of security and surveillance applications.
Editor's Introduction: Humanities in the Loop

Abstract This editor's introduction welcomes readers to a new interdisciplinary undertaking. The community of practice Critical AI addresses hopes to bring critical thinking of the kind that interpretive disciplines foster into dialogue with work by technologists and others who share the understanding of interdisciplinary research as a powerful tool for building accountable technology in the public interest. Critical AI studies aims to shape and activate conversations in academia, industry, policymaking, media, and the public at large. The long and ongoing history of “AI,” including the data-driven technologies that now claim that name, remains riddled by three core dilemmas: (1) reductive and controversial meanings of “intelligence”; (2) problematic benchmarks and tests for supposedly scientific terms such as “AGI”; and (3) bias, errors, stereotypes, and concentration of power. AI hype today is steeped in blends of utopian and dystopian discourse that distract from the real-world harms of existing technologies. In reality, what is hyped and anthropomorphized as “AI” and even “AGI” is the product not only of technology companies and investors but also—and more fundamentally—of the many millions of people and communities subject to copyright infringement, nonconsensual use of data, bias, environmental harms, and the low-wage and high-stress modes of “human in the loop” through which systems for probabilistic mimicry improve their performance in an imitation game.
Iceberg Hashing: Optimizing Many Hash-Table Criteria at Once

Despite being one of the oldest data structures in computer science, hash tables continue to be the focus of a great deal of both theoretical and empirical research. A central reason for this is that many of the fundamental properties that one desires from a hash table are difficult to achieve simultaneously; thus many variants offering different trade-offs have been proposed. This article introduces Iceberg hashing, a hash table that simultaneously offers the strongest known guarantees on a large number of core properties. Iceberg hashing supports constant-time operations while improving on the state of the art for space efficiency, cache efficiency, and low failure probability. Iceberg hashing is also the first hash table to support a load factor of up to 1 - o(1) while being stable, meaning that the position where an element is stored only ever changes when resizes occur. In fact, in the setting where keys are Θ (log n ) bits, the space guarantees that Iceberg hashing offers, namely that it uses at most \(\log \binom{|U|}{n} + O(n \log \ \text{log} n)\) bits to store n items from a universe U , matches a lower bound by Demaine et al. that applies to any stable hash table. Iceberg hashing introduces new general-purpose techniques for some of the most basic aspects of hash-table design. Notably, our indirection-free technique for dynamic resizing, which we call waterfall addressing, and our techniques for achieving stability and very-high probability guarantees, can be applied to any hash table that makes use of the front-yard/backyard paradigm for hash table design.