Reviewerly/exHarmony huggingface dataset:
One-shot color mapping of a ray direction field for obtaining three-dimensional profiles integrating deep neural networks

A method for simultaneously and instantly obtaining both a three-dimensional (3D) surface and its inclination angle distribution from a single image captured by an imaging system equipped with a coaxial multicolor filter that integrates deep neural networks (DNNs) is proposed. The imaging system can obtain a light-ray direction in the field of view through one-shot color mapping. Light rays reflected from a 3D surface, even if it has microscale height variations with a small inclination angle distribution, can be assigned different colors depending on their directions by the imaging system. This enables the acquisition of the surface inclination angle distribution. Assuming a smooth and continuous 3D surface, it is possible to reconstruct the surface from a single captured image using DNNs. The DNNs can provide the height variations of the 3D surface by solving a nonlinear partial differential equation that represents the relationship between height variation and the direction of light rays. This method is validated analytically and experimentally using microscale convex surfaces.
Person-action Instance Search in Story Videos: An Experimental Study

Person-Action instance search (P-A INS) aims to retrieve the instances of a specific person doing a specific action, which appears in the 2019–2021 INS tasks of the world-famous TREC Video Retrieval Evaluation (TRECVID). Most of the top-ranking solutions can be summarized with a Division-Fusion-Optimization (DFO) framework, in which person and action recognition scores are obtained separately, then fused, and, optionally, further optimized to generate the final ranking. However, TRECVID only evaluates the final ranking results, ignoring the effects of intermediate steps and their implementation methods. We argue that conducting the fine-grained evaluations of intermediate steps of DFO framework will (1) provide a quantitative analysis of the different methods’ performance in intermediate steps; (2) find out better design choices that contribute to improving retrieval performance; and (3) inspire new ideas for future research from the limitation analysis of current techniques. Particularly, we propose an indirect evaluation method motivated by the leave-one-out strategy, which finds an optimal solution surpassing the champion teams in 2020–2021 INS tasks. Moreover, to validate the generalizability and robustness of the proposed solution under various scenarios, we specifically construct a new large-scale P-A INS dataset and conduct comparative experiments with both the leading NIST TRECVID INS solution and the state-of-the-art P-A INS method. Finally, we discuss the limitations of our evaluation work and suggest future research directions.
Differentially Private Methods for Releasing Results of Stability Analyses

AbstractData stewards and analysts can promote transparent and trustworthy science and policy-making by facilitating assessments of the sensitivity of published results to alternate analysis choices. For example, researchers may want to assess whether the results change substantially when different subsets of data points (e.g., sets formed by demographic characteristics) are used in the analysis, or when different models (e.g., with or without log transformations) are estimated on the data. Releasing the results of such stability analyses leaks information about the data subjects. When the underlying data are confidential, the data stewards and analysts may seek to bound this information leakage. We present methods for stability analyses that can satisfy differential privacy, a definition of data confidentiality providing such bounds. We use regression modeling as the motivating example. The basic idea is to split the data into disjoint subsets, compute a measure summarizing the difference between the published and alternative analysis on each subset, aggregate these subset estimates, and add noise to the aggregated value to satisfy differential privacy. We illustrate the methods using regressions in which an analyst compares coefficient estimates for different groups in the data, and in which analysts fit two different models on the data.KEYWORDS: ConfidentialityRegressionReplicabilityReproducibilityVerification Supplementary MaterialsThe supplementary material includes results of simulation studies that illustrate the performance of the differentially private AD measures.Disclosure StatementThe authors report there are no competing interests to declare.Additional informationFundingThis work is supported by a grant from the National Science Foundation (SES-2214756).
Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching

As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods.
MTGL40-5: A Multi-Temporal Dataset for Remote Sensing Image Geo-Localization

Image-based geo-localization focuses on predicting the geographic information of query images by matching them with annotated images in a database. To facilitate relevant studies, researchers collect numerous images to build the datasets, which explore many challenges faced in real-world geo-localization applications, significantly improving their practicability. However, a crucial challenge that often arises is overlooked, named the cross-time challenge in this paper, i.e., if query and database images are taken from the same landmark but at different time periods, the significant difference in their image content caused by the time gap will notably increase the difficulty of image matching, consequently reducing geo-localization accuracy. The cross-time challenge has a greater negative influence on non-real-time geo-localization applications, particularly those involving a long time span between query and database images, such as satellite-view geo-localization. Furthermore, the rough geographic information (e.g., names) instead of precise coordinates provided by most existing datasets limits the geo-localization accuracy. Therefore, to solve these problems, we propose a dataset, MTGL40-5, which contains remote sensing (RS) satellite images captured from 40 large-scale geographic locations spanning five different years. These large-scale images are split to create query images and a database with landmark labels for geo-localization. By observing images from the same landmark but at different time periods, the cross-time challenge becomes more evident. Thus, MTGL40-5 supports researchers in tackling this challenge and further improving the practicability of geo-localization. Moreover, it provides additional geographic coordinate information, enabling the study of high-accuracy geo-localization. Based on the proposed MTGL40-5 dataset, many existing geo-localization methods, including state-of-the-art approaches, struggle to produce satisfactory results when facing the cross-time challenge. This highlights the importance of proposing MTGL40-5 to address the limitations of current methods in effectively solving the cross-time challenge.
Remote Sensing Image Dehazing through an Unsupervised Generative Adversarial Network

The degradation of visual quality in remote sensing images caused by haze presents significant challenges in interpreting and extracting essential information. To effectively mitigate the impact of haze on image quality, we propose an unsupervised generative adversarial network specifically designed for remote sensing image dehazing. This network includes two generators with identical structures and two discriminators with identical structures. One generator is focused on image dehazing, while the other generates images with added haze. The two discriminators are responsible for distinguishing whether an image is real or generated. The generator, employing an encoder-decoder architecture, is designed based on the proposed multi-scale feature-extraction modules and attention modules. The proposed multi-scale feature-extraction module, comprising three distinct branches, aims to extract features with varying receptive fields. Each branch comprises dilated convolutions and attention modules. The proposed attention module includes both channel and spatial attention components. It guides the feature-extraction network to emphasize haze and texture within the remote sensing image. For enhanced generator performance, a multi-scale discriminator is also designed with three branches. Furthermore, an improved loss function is introduced by incorporating color-constancy loss into the conventional loss framework. In comparison to state-of-the-art methods, the proposed approach achieves the highest peak signal-to-noise ratio and structural similarity index metrics. These results convincingly demonstrate the superior performance of the proposed method in effectively removing haze from remote sensing images.
Mechanising Gödel–Löb Provability Logic in HOL Light

Abstract We introduce our implementation in HOL Light of the metatheory for Gödel–Löb provability logic (GL), covering soundness and completeness w.r.t. possible world semantics and featuring a prototype of a theorem prover for GL itself. The strategy we develop here to formalise the modal completeness proof overcomes the technical difficulty due to the non-compactness of GL and is an adaptation—according to the formal language and tools at hand—of the proof given in George Boolos’ 1995 monograph. Our theorem prover for GL relies then on this formalisation, is implemented as a tactic of HOL Light that mimics the proof search in the labelled sequent calculus $$\textsf{G3KGL}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>G</mml:mi><mml:mn>3</mml:mn><mml:mi>KGL</mml:mi></mml:mrow></mml:math> , and works as a decision algorithm for the provability logic: if the algorithm positively terminates, the tactic succeeds in producing a HOL Light theorem stating that the input formula is a theorem of GL; if the algorithm negatively terminates, the tactic extracts a model falsifying the input formula. We discuss our code for the formal proof of modal completeness and the design of our proof search algorithm. Furthermore, we propose some examples of the latter’s interactive and automated use.
On Informatics Approaches to Overcoming Natural Science Crisis

Husserl told readers that there are two ways to understand his theory, namely “starting from the world of life” or “starting from psychology”. We believe that theoretical informatics and information philosophy can be combined with the Husserl phenomenological movement as two important options to overcome the crisis of contemporary natural science. The purposes of this article are to analyze the ontological and epistemological roots of the scientific “crisis”; to criticize the one-sidedness of the mainstream of Western science and philosophy over the past 2300 years; to develop and perfect the foundation and core of information science: “theoretical informatics”; to apply the experience of “Information Psychology” to the “discipline informatization” in Law, Ethics, Linguistics, etc.; and to abstract information ontology, epistemology, and axiology based on theoretical informatics.
Use of novel artificial intelligence computer-assisted detection (AI-CAD) for screening mammography: an analysis of 17,884 consecutive two-view full-field digital mammography screening exams

Novel artificial intelligence computer-assisted detection (AI-CAD) systems based on deep learning (DL) promise to support screen reading.To test a DL-AI-CAD system compared to human reading on consecutive screening mammograms.In this retrospective study, 17,884 consecutive anonymized screening mammograms, double-read from January to November 2018, were processed by the DL-AI-CAD system. AI-CAD reading was considered positive if the AI-CAD case scores exceeded 30 (range = 1-100) and the lesion was correctly marked. Likewise, human reading (R1 or R2, respectively) was considered positive if the lesion was correctly identified and called. Receiver operating characteristic (ROC) analysis was performed and accuracy data were calculated. Ground truth for benign lesions: absence of malignancy after cancer registry matching (2022); for malignancy: histopathologic proof; evaluation was patient-based.In total, 114 screen-detected and 17 interval cancers (ICA) occurred. ROC analysis of screen-detected cancers yielded an AUC of 89% for AI-CAD. Sensitivity/specificity was 81.7%/80.2% for AI-CAD; 77.1%/91.7% for R1; 78.6/91.6% for R2. Combining each human reading with AI-CAD was as sensitive as human double-reading (all approximately 88%), but less specific (approximately 75%) compared to human double-reading (approximately 87%). These AI-CAD combinations required consensus readings for twice as many cases as the human combination. Four of 17 ICA exceeded a case score of 30; two of four CAD correctly marked the quadrant of the subsequent ICA.Including ICA cases, this AI-CAD achieved comparable sensitivity to human reading at lower specificity. Combining human reading and AI-CAD allows increasing sensitivity compared to single-reading.
Detection of Streaks in Astronomical Images Using Machine Learning

Satellites in Low Earth Orbit (LEO) pose a challenge to astronomy observations requiring long exposure times or wide observation areas. As the number of satellites in LEO dramatically increases, it motivates an increased need for methods to filter out artifacts caused by satellites crossing into observation fields. This paper develops and evaluates a deep learning model based on U-Net to filter these artifacts from collected data. The proposed model is compared with two existing filtering methods on a dataset generated using the state-of-the-art tool Pyradon. Although the initial application of deep learning does include some unpredictable behavior not found in traditional algorithms, the proposed model outperforms the existing methods in overall accuracy while requiring significantly less computational time. This suggests that the application of deep learning to satellite artifact removal which has previously been underdeveloped in the literature may be an appropriate avenue.
YOLO-CID: Improved YOLOv7 for X-ray Contraband Image Detection

Currently, X-ray inspection systems may produce false detections due to factors such as the varying sizes of contraband images, complex backgrounds, and blurred edges. To address this issue, we propose the YOLO-CID method for contraband image detection. Firstly, we designed the MP-OD module in the backbone network to enhance the model’s ability to extract key information from complex background images. Secondly, at the neck of the network, we designed a simplified version of BiFPN to add cross-scale connection lines in the feature fusion structure, to preserve deeper semantic information and enhance the network’s ability to represent objects in low-contrast or occlusion situations. Finally, we added a new object detection layer to improve the model’s accuracy in detecting small objects in dense environments. Experimental results on the PIDray public dataset show that the average accuracy rate of the YOLO-CID algorithm is 82.7% and the recall rate is 81.2%, which are 4.9% and 3.2% higher than the YOLOv7 algorithm, respectively. At the same time, the mAP on the CLCXray dataset reached 80.2%. Additionally, it can achieve a real-time detection speed of 40 frames per second and 43 frames per second in real scenes. These results demonstrate the effectiveness of the YOLO-CID algorithm in X-ray contraband detection.
Near-Term Efficient Quantum Algorithms for Entanglement Analysis

Entanglement plays a crucial role in quantum physics and is the key resource in quantum information processing. However, entanglement detection and quantification are believed to be hard due to the operational impracticality of existing methods. This work proposes three near-term efficient algorithms that exploit the hybrid quantum-classical technique to address this difficulty. The first algorithm finds the Schmidt decomposition---a powerful tool to analyze the properties and structure of entanglement---for bipartite pure states. While the logarithm negativity can be calculated from the Schmidt decomposition, we propose the second algorithm to estimate the logarithm negativity for bipartite pure states, where the width of the parameterized quantum circuits is further reduced. Finally, we generalize our framework for mixed states, leading to our third algorithm that detects entanglement on specific families of states, and determines distillability in general. All three algorithms share a similar framework where the optimizations are accomplished by maximizing a cost function utilizing local parameterized quantum circuits, with better hardware efficiency and practicality compared to existing methods. The experimental implementation on Quantum Leaf using the Institute of Physics, Chinese Academy of Sciences superconducting quantum processor exhibits the validity and practicality of our methods for analyzing and quantifying entanglement on near-term quantum devices.
PSI Analysis of Adversarial-Attacked DCNN Models

In the past few years, deep convolutional neural networks (DCNNs) have surpassed human performance in tasks related to recognizing objects. However, DCNNs are also threatened by performance degradation due to adversarial examples. DCNNs are essentially black-boxed, and it is not known how the output is determined internally; consequently, it is not known how adversarial attacks cause performance degradation inside the DCNNs. To observe the internal neuronal activities of DCNN models for adversarial examples, we analyzed the population sparseness index (PSI) values at each layer of two representative DCNN models, namely AlexNet and VGG11. From the experimental results, we observed that the internal responses of the two DCNN models to adversarial examples exhibited distinct layer-wise PSI values, differing from the internal responses to benign examples. The main contribution of this study is the discovery of significant differences in the internal responses of two specific DCNN models to adversarial and benign examples by PSI. Furthermore, our research has the potential not only to contribute to the design of more robust DCNN models against adversarial examples but also to bridge the gap between the fields of artificial intelligence and neurophysiology of the brain.
PVTReID: A Quick Person Reidentification-Based Pyramid Vision Transformer

Person re-identification (ReID) has attracted the attention of a large number of researchers due to its wide range of applications. However, due to the difficulty of extracting robust features and the complexity of the feature extraction process, ReID is difficult to truly apply in practice. In this paper, we utilize Pyramid Vision Transformer (PVT) as the backbone for feature extraction and propose a PVT-based ReID method in conjunction with other studies. First, we establish a basic model using powerful methods verified on CNN-based ReID. Second, to further improve the robustness of the features extracted from the PVT backbone, we design two new modules: (1) a local feature clustering (LFC) module is used to select the most discrete local features and cluster them individually by calculating the distance between local and global features, and (2) side information embeddings (SIE) are used to encode nonvisual information and send it to the network for use training in order to reduce its impact on the features. Our experiments show that the proposed PVTReID achieves an mAP of 63.2% on MSMT17 and 80.5% on DukeMTMC-reID. In addition, we evaluated the inference speed for images achieved by different methods, proving that image inference is faster with our proposed method. These results clearly illustrate that using PVT as a backbone network with LFC and SIE modules can improve inference speed while extracting robust features.
Enhancing Sensor-Based Mobile User Authentication in a Complex Environment by Deep Learning

With the advent of smart mobile devices, end users get used to transmitting and storing their individual privacy in them, which, however, has aroused prominent security concerns inevitably. In recent years, numerous researchers have primarily proposed to utilize motion sensors to explore implicit authentication techniques. Nonetheless, for them, there are some significant challenges in real-world scenarios. For example, depending on the expert knowledge, the authentication accuracy is relatively low due to some difficulties in extracting user micro features, and noisy labels in the training phrase. To this end, this paper presents a real-time sensor-based mobile user authentication approach, ST-SVD, a semi-supervised Teacher–Student (TS) tri-training algorithm, and a system with client–server (C-S) architecture. (1) With S-transform and singular value decomposition (ST-SVD), we enhance user micro features by transforming time-series signals into 2D time-frequency images. (2) We employ a Teacher–Student Tri-Training algorithm to reduce label noise within the training sets. (3) To obtain a set of robust parameters for user authentication, we input the well-labeled samples into a CNN (convolutional neural network) model, which validates our proposed system. Experimental results on large-scale datasets show that our approach achieves authentication accuracy of 96.32%, higher than the existing state-of-the-art methods.
Investigating Self-Rationalizing Models for Commonsense Reasoning

The rise of explainable natural language processing spurred a bulk of work on datasets augmented with human explanations, as well as technical approaches to leverage them. Notably, generative large language models offer new possibilities, as they can output a prediction as well as an explanation in natural language. This work investigates the capabilities of fine-tuned text-to-text transfer Transformer (T5) models for commonsense reasoning and explanation generation. Our experiments suggest that while self-rationalizing models achieve interesting results, a significant gap remains: classifiers consistently outperformed self-rationalizing models, and a substantial fraction of model-generated explanations are not valid. Furthermore, training with expressive free-text explanations substantially altered the inner representation of the model, suggesting that they supplied additional information and may bridge the knowledge gap. Our code is publicly available, and the experiments were run on open-access datasets, hence allowing full reproducibility.