Reviewerly/exHarmony huggingface dataset:
Understanding Masked Autoencoders via Hierarchical Latent Variable Models

Masked autoencoder (MAE), a simple and effective self-supervised learning framework based on the reconstruction of masked image regions, has recently achieved prominent success in a variety of vision tasks. Despite the emergence of intriguing empirical observations on MAE, a theoretically principled understanding is still lacking. In this work, we formally characterize and justify existing empirical in-sights and provide theoretical guarantees of MAE. We formulate the underlying data-generating process as a hierarchical latent variable model, and show that under reasonable assumptions, MAE provably identifies a set of latent variables in the hierarchical model, explaining why MAE can extract high-level information from pixels. Further, we show how key hyperparameters in MAE (the masking ratio and the patch size) determine which true latent variables to be recovered, therefore influencing the level of semantic information in the representation. Specifically, extremely large or small masking ratios inevitably lead to low-level representations. Our theory offers coherent explanations of existing empirical observations and provides insights for potential empirical improvements and fundamental limitations of the masked-reconstruction paradigm. We conduct extensive experiments to validate our theoretical insights.
Meta-Learning with a Geometry-Adaptive Preconditioner

Model-agnostic meta-learning (MAML) is one of the most successful meta-learning algorithms. It has a bi-level optimization structure where the outer-loop process learns a shared initialization and the inner-loop process optimizes task-specific weights. Although MAML relies on the standard gradient descent in the inner-loop, recent studies have shown that controlling the inner-loop's gradient descent with a meta-learned preconditioner can be beneficial. Existing preconditioners, however, cannot simultaneously adapt in a task-specific and path-dependent way. Additionally, they do not satisfy the Riemannian metric condition, which can enable the steepest descent learning with preconditioned gradient. In this study, we propose Geometry-Adaptive Preconditioned gradient descent (GAP) that can overcome the limitations in MAML; GAP can efficiently meta-learn a preconditioner that is dependent on task-specific parameters, and its preconditioner can be shown to be a Riemannian metric. Thanks to the two properties, the geometry-adaptive preconditioner is effective for improving the inner-loop optimization. Experiment results show that GAP outperforms the state-of-the-art MAML family and preconditioned gradient descent-MAML (PGD-MAML) family in a variety of few-shot learning tasks. Code is available at: https://github.com/Suhyun777/CVPR23-GAP.
BOOM! Tephrochronological dataset and exploration tool of the Southern (33–46° S) and Austral (49–55° S) volcanic zones of the Andes

Tephrochronology studies the deposits of explosive volcanic eruptions in the stratigraphic record. The Southern (SVZ, 33–46° S) and Austral (AVZ, 49–55° S) Volcanic Zones of the Andes are two very active volcanic zones where tephrochronology is of great use. There, it can be used to improve chronologies of paleoenvironmental records in Patagonia, an area providing valuable records at global scale; as well as to identify areas likely to be affected by volcanic eruptions in the future, essential for producing volcanic hazard maps. The close proximity of many volcanic centers with recurrent explosive activity, which have very similar geochemical compositions, and very often poor age constraints, represent a challenge for the study of tephrochronology in the region. In addition to this, the ever-growing amount of tephrochronological information in the area, dispersed in different types of publications which vary greatly in format, makes the integration of the data produced by different actors, and consecutively its interpretation, increasingly difficult. Here we address this issue by compiling the BOOM! dataset, which integrates ∼30 years of research on 32 active volcanic centers and 132 different eruptions, which took place during the last 20,000 years. To help users find and reuse data in the large dataset, we developed an online platform which provides user-friendly tools for exploring it, and helps users download subsets of it. To integrate this very heterogeneous information, special attention was given to include information which allows users to evaluate data quality and comparability, as well as to provide tools in the explorer for users to filter data by different criteria. The integration of this dataset opens new perspectives for the development of novel visualizations of tephrochronological data, for example, to better understand the multidimensional uncertainties associated with it. For example, uncertainties associated with analytical precision, with age estimates of both tephra deposits and volcanic eruptions, and of tephra classification. Additionally, it allows for the use of robust statistical tools to correlate tephra deposits, including those based on machine learning algorithms, which are here explored.
The Role of Information Technology in Improving Urban Governance

The rapid development of information technology has a significant impact on various fields of human life, including urban governance. In the context of urban governance, information technology can be used to improve efficiency, transparency, and community participation in the decision-making process that has an impact on the quality of life of city residents. The purpose of this research is to explain the role of information technology in improving urban governance because it is important in the context of increasingly complex urban development and requires appropriate and efficient decision-making. The focus of this research is qualitative. Methods for gathering information included paying close attention and taking detailed notes, with subsequent analysis including data reduction, visualisation, and inference. The study arrived at the conclusion that the role of information technology in improving urban governance is crucial and requires special attention from the government and relevant stakeholders. By improving access and quality of information technology services, enhancing digital literacy and information technology skills, increasing community participation in urban governance, and improving data security and privacy, the application of information technology in urban governance can become more effective and efficient in improving urban governance.&#x0D;
A Multi-Stain Breast Cancer Histological Whole-Slide-Image Data Set from Routine Diagnostics

Abstract The analysis of FFPE tissue sections stained with haematoxylin and eosin (H&amp;E) or immunohistochemistry (IHC) is essential for the pathologic assessment of surgically resected breast cancer specimens. IHC staining has been broadly adopted into diagnostic guidelines and routine workflows to assess the status of several established biomarkers, including ER, PGR, HER2 and KI67. Biomarker assessment can also be facilitated by computational pathology image analysis methods, which have made numerous substantial advances recently, often based on publicly available whole slide image (WSI) data sets. However, the field is still considerably limited by the sparsity of public data sets. In particular, there are no large, high quality publicly available data sets with WSIs of matching IHC and H&amp;E-stained tissue sections from the same tumour. Here, we publish the currently largest publicly available data set of WSIs of tissue sections from surgical resection specimens from female primary breast cancer patients with matched WSIs of corresponding H&amp;E and IHC-stained tissue, consisting of 4,212 WSIs from 1,153 patients.
Mobile superconducting strip photon detection system with efficiency over 70% at 1550 nm wavelength

We developed a mobile superconducting strip photon detector (SSPD) system operated in a liquid-helium Dewar. By adopting highly disordered NbTiN thin films, we successfully enhanced the detection performance of superconducting strips at higher operation temperatures and realized SSPDs with nearly saturated detection efficiency at 4.2 K. Then we customized a compact liquid-helium Dewar and a battery-based electronic module to minimize the SSPD system. A mobile SSPD system was integrated, which showed a system detection efficiency of 72% for a 1550 nm wavelength with a dark count rate of 200 cps and a timing jitter of 67.2 ps. The system has a weight of 40 kg and a power consumption of 500 mW, which can work continuously for 20 hours. The metrics can be further optimized in accordance with the various practical application platforms, such as aircraft, drones, etc.
B2C-AFM: Bi-Directional Co-Temporal and Cross-Spatial Attention Fusion Model for Human Action Recognition

Human Action Recognition plays a driving engine of many human-computer interaction applications. Most current researches focus on improving the model generalization by integrating multiple homogeneous modalities, including RGB images, human poses, and optical flows. Furthermore, contextual interactions and out-of-context sign languages have been validated to depend on scene category and human per se. Those attempts to integrate appearance features and human poses have shown positive results. However, with human poses’ spatial errors and temporal ambiguities, existing methods are subject to poor scalability, limited robustness, and sub-optimal models. In this paper, inspired by the assumption that different modalities may maintain temporal consistency and spatial complementarity, we present a novel Bi-directional Co-temporal and Cross-spatial Attention Fusion Model (B2C-AFM). Our model is characterized by the asynchronous fusion strategy of multi-modal features along temporal and spatial dimensions. Besides, the novel explicit motion-oriented pose representations called Limb Flow Fields (Lff) are explored to alleviate the temporal ambiguity regarding human poses. Experiments on publicly available datasets validate our contributions. Abundant ablation studies experimentally show that B2C-AFM achieves robust performance across seen and unseen human actions. The codes are available here <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Reinforcing Industry 4.0 With Digital Twins and Blockchain-Assisted Federated Learning

The Internet of Things (IoT) has revolutionized the manufacturing process in the industry. It has created a new ecosystem allowing a diversified set of devices to be controlled remotely with minimal human intervention. Today, with the advances in intelligence, processing, storage, communication, and networking capabilities of IoT devices, we are one step closer to realizing the vision of Industry 4.0. Cyber-physical systems (CPS) are now significantly more intelligent and automated with the aid of advances in Machine Learning (ML). Intelligent IoT (IIoT), Digital Twins (DT) and the advances in mobile networks are now paving the path towards decentralized self-managed CPS in the industry. DT permits mobile networks to provide adaptive and dynamic configurations for cooperative CPS. Moreover, trustworthy cooperation may be realized with blockchain. In this article, we present a blockchain-assisted hierarchical federated learning (FL)-enabled platform (HFL) for Industry 4.0. The solution integrates DT into CPS to accurately capture the characteristics of industrial IoT devices and assist in the HFL process. A two-stage FL algorithm is used that groups Internet-enabled factory machinery and their DTs into groups in accordance with their organizational structure. A global model is created for the groups from the averaged local models and the DT model in the first stage. During the second stage, federated aggregation is used to create a global model from the first-stage models. Blockchain is used to cross-verify and validate newly added blocks with the support of validator nodes. Numerical analysis is performed to compare between the presented DT-enabled and blockchain-assisted HFL solution and benchmark solutions in terms of network overhead, block optimization, and accuracy.
CloudpredNet: An Ultra-Short-Term Movement Prediction Model for Ground-Based Cloud Image

Ground-based cloud images can provide information on weather and cloud conditions, which are important for cloud monitoring and PV power generation forecasting. Prediction of short-time cloud movement from images is a major means of intra-hourly irradiation forecasting for solar power generation and is also important for precipitation forecasting. However, there is a lack of advanced and complete methods for cloud movement prediction from ground-based cloud images, and traditional techniques based on image processing and motion vector calculations have difficulty in predicting cloud morphological changes, which makes accurate prediction of cloud motion (especially nonlinear motion) very challenging. Therefore, this paper proposes CloudpredNet, a ground-based cloud ultra-short-term movement prediction model based on an "encoder-generator" architecture. This paper also proposes a loss function dedicated to the time series prediction of ground-based cloud images and combines the attention mechanism to train the model. The model is validated on a publicly available dataset, and it is demonstrated that it has good performance in all metrics of cloud image generation for the next 10 minutes.
Subsurface Lithology Classification Using Well Log Data, an Application of Supervised Machine Learning

An attempt has been made to predict the subsurface lithology of the adjunct exploratory well drilled for hydrocarbon exploration. Both, decision trees and random forests algorithms of supervised machine learning are used for the comparison and prediction of the subsurface lithologies. It is generally observed that at each stage, the decision tree looks for the locally optimal alternative, whereas the random forest in contrast, makes use of many decision trees, which has a lower danger of overfitting. The random forest uses the predictions from all of the decision trees as opposed to just one and makes final predictions based on the majority of variables. Overall, the random forest is essentially a collection of decision trees, the output of which is combined to produce a single overall result. In the current research, the models are developed and trained to predict the subsurface lithology of the adjacent well with ~ 9000 well log data points, 70% of which are used for training and 30% for testing. Finally, the accuracy check using the F1-score is performed, showing 97 and 94% accuracy in subsurface lithology prediction for random forest and decision tree classifications, respectively.
Curved Geometric Networks for Visual Anomaly Recognition

Learning a latent embedding to understand the underlying nature of data distribution is often formulated in Euclidean spaces with zero curvature. However, the success of the geometry constraints, posed in the embedding space, indicates that curved spaces might encode more structural information, leading to better discriminative power and hence richer representations. In this work, we investigate the benefits of the curved space for analyzing anomalous, open-set, or out-of-distribution (OOD) objects in data. This is achieved by considering embeddings via three geometry constraints, namely, spherical geometry (with positive curvature), hyperbolic geometry (with negative curvature), or mixed geometry (with both positive and negative curvatures). Three geometric constraints can be chosen interchangeably in a unified design, given the task at hand. Tailored for the embeddings in the curved space, we also formulate functions to compute the anomaly score. Two types of geometric modules (i.e., geometric-in-one (GiO) and geometric-in-two (GiT) models) are proposed to plug in the original Euclidean classifier, and anomaly scores are computed from the curved embeddings. We evaluate the resulting designs under a diverse set of visual recognition scenarios, including image detection (multiclass OOD detection and one-class anomaly detection) and segmentation (multiclass anomaly segmentation and one-class anomaly segmentation). The empirical results show the effectiveness of our proposal through consistent improvement over various scenarios. The code is made available at https://github.com/JHome1/GiO-GiT.
Retrieval-augmented Image Captioning

Inspired by retrieval-augmented language generation and pretrained Vision and Language (V&L) encoders, we present a new approach to image captioning that generates sentences given the input image and a set of captions retrieved from a datastore, as opposed to the image alone. The encoder in our model jointly processes the image and retrieved captions using a pretrained V&L BERT, while the decoder attends to the multimodal encoder representations, benefiting from the extra textual evidence from the retrieved captions. Experimental results on the COCO dataset show that image captioning can be effectively formulated from this new perspective. Our model, named EXTRA, benefits from using captions retrieved from the training dataset, and it can also benefit from using an external dataset without the need for retraining. Ablation studies show that retrieving a sufficient number of captions (e.g., k=5) can improve captioning quality. Our work contributes towards using pretrained V&L encoders for generative tasks, instead of standard classification tasks.
Evaluating the Robustness of Discrete Prompts

Discrete prompts have been used for fine-tuning Pre-trained Language Models for diverse NLP tasks. In particular, automatic methods that generate discrete prompts from a small set of training instances have reported superior performance. However, a closer look at the learnt prompts reveals that they contain noisy and counter-intuitive lexical constructs that would not be encountered in manually-written prompts. This raises an important yet understudied question regarding the robustness of automatically learnt discrete prompts when used in downstream tasks. To address this question, we conduct a systematic study of the robustness of discrete prompts by applying carefully designed perturbations into an application using AutoPrompt and then measure their performance in two Natural Language Inference (NLI) datasets. Our experimental results show that although the discrete prompt-based method remains relatively robust against perturbations to NLI inputs, they are highly sensitive to other types of perturbations such as shuffling and deletion of prompt tokens. Moreover, they generalize poorly across different NLI datasets. We hope our findings will inspire future work on robust discrete prompt learning.
Analysis of structural effects of sickle cell disease on brain vasculature of mice using three-dimensional quantitative phase imaging

SignificanceAlthough the molecular origins of sickle cell disease (SCD) have been extensively studied, the effects of SCD on the vasculature—which can influence blood clotting mechanisms, pain crises, and strokes—are not well understood. Improving this understanding can yield insight into the mechanisms and wide-ranging effects of this devastating disease.AimWe aim to demonstrate the ability of a label-free 3D quantitative phase imaging technology, called quantitative oblique back-illumination microscopy (qOBM), to provide insight into the effects of SCD on brain vasculature.ApproachUsing qOBM, we quantitatively analyze the vasculature of freshly excised, but otherwise unaltered, whole mouse brains. We use Townes sickle transgenic mice, which closely recapitulate the pathophysiology of human SCD, and sickle cell trait mice as controls. Two developmental time points are studied: 6-week-old mice and 20-week-old mice. Quantitative structural and biophysical parameters of the vessels (including the refractive index (RI), which is linearly proportional to dry mass) are extracted from the high-resolution images and analyzed.ResultsqOBM reveals structural differences in the brain blood vessel thickness (thinner for SCD in particular brain regions) and the RI of the vessel wall (higher and containing a larger variation throughout the brain for SCD). These changes were only significant in 20-week-old mice. Further, vessel breakages are observed in SCD mice at both time points. The vessel wall RI distribution near these breaks, up to 350 μm away from the breaking point, shows an erratic behavior characterized by wide RI variations. Vessel diameter, tortuosity, texture within the vessel, and structural fractal patterns are found to not be statistically different. As with vessel breaks, we also observe blood vessel blockages only in mice brains with SCD.ConclusionsqOBM provides insight into the biophysical and structural composition of brain blood vessels in mice with SCD. Data suggest that the RI may be an indirect indicator of vessel rigidity, vessel strength, and/or tensions, which change with SCD. Future ex vivo and in vivo studies with qOBM could improve our understanding of SCD.
Transformer-based rapid human pose estimation network

Most current human pose estimation methods pursue excellent performance via large models and intensive computational requirements, resulting in slower models. These methods cannot be effectively adopted for human pose estimation in real applications due to their high memory and computational costs. To achieve a trade-off between accuracy and efficiency, we propose TRPose, a Transformer-based network for human pose estimation rapidly. TRPose consists of an early convolutional stage and a later Transformer stage seamlessly. Concretely, the convolutional stage forms a Rapid Fusion Module (RFM), which efficiently acquires multi-scale features via three parallel convolution branches. The Transformer stage utilizes multi-resolution Transformers to construct a Dual scale Encoder Module (DEM), aiming at learning long-range dependencies from different scale features of the whole human skeletal keypoints. The experiments show that TRPose acquires 74.3 AP and 73.8 AP on COCO validation and testdev datasets with 170+ FPS on a GTX2080Ti, which achieves the better efficiency and effectiveness trade-offs than most state-of-the-art methods. Our model also outperforms mainstream Transformer-based architectures on MPII dataset, yielding 89.9 PCK@0.5 score on val set without extra data.
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages

While there has been a recent burgeoning of applications at the intersection of natural and programming languages, such as code generation and code summarization, these applications are usually English-centric. This creates a barrier for program developers who are not proficient in English. To mitigate this gap in technology development across languages, we propose a multilingual dataset, MCoNaLa, to benchmark code generation from natural language commands extending beyond English. Modeled off of the methodology from the English Code/Natural Language Challenge (CoNaLa) dataset, we annotated a total of 896 NL-Code pairs in three languages: Spanish, Japanese, and Russian. We present a systematic evaluation on MCoNaLa by testing state-of-the-art code generation systems. Although the difficulties vary across three languages, all systems lag significantly behind their English counterparts, revealing the challenges in adapting code generation to new languages.