Reviewerly/exHarmony huggingface dataset:
LaTr: Layout-Aware Transformer for Scene-Text VQA

We propose a novel multimodal architecture for Scene Text Visual Question Answering (STVQA), named Layout-Aware Transformer (LaTr). The task of STVQA requires models to reason over different modalities. Thus, we first investigate the impact of each modality, and reveal the importance of the language module, especially when enriched with layout information. Accounting for this, we propose a single objective pre-training scheme that requires only text and spatial cues. We show that applying this pre-training scheme on scanned documents has certain advantages over using natural images, despite the domain gap. Scanned documents are easy to procure, text-dense and have a variety of layouts, helping the model learn various spatial cues (e.g. left-of, below etc.) by tying together language and layout information. Compared to existing approaches, our method performs vocabulary-free decoding and, as shown, generalizes well beyond the training vocabulary. We further demonstrate that LaTr improves robustness towards OCR errors, a common reason for failure cases in STVQA. In addition, by leveraging a vision transformer, we eliminate the need for an external object detector. LaTr outperforms state-of-the-art STVQA methods on multiple datasets. In particular, +7.6% on TextVQA, +10.8% on ST-VQA and +4.0% on OCR-VQA (all absolute accuracy numbers).
Supplemental Breast Cancer Screening in Women with Dense Breasts and Negative Mammography: A Systematic Review and Meta-Analysis

Background The best supplemental breast cancer screening modality in women at average risk or intermediate risk for breast cancer with dense breast and negative mammogram remains to be determined. Purpose To conduct systematic review and meta-analysis comparing clinical outcomes of the most common available supplemental screening modalities in women at average risk or intermediate risk for breast cancer in patients with dense breasts and mammography with negative findings. Materials and Methods A comprehensive search was conducted until March 12, 2020, in Medline, Epub Ahead of Print and In-Process and Other Non-Indexed Citations; Embase Classic and Embase; Cochrane Central Register of Controlled Trials; and Cochrane Database of Systematic Reviews, for Randomized Controlled Trials and Prospective Observational Studies. Incremental cancer detection rate (CDR); positive predictive value of recall (PPV1); positive predictive value of biopsies performed (PPV3); and interval CDRs of supplemental imaging modalities, digital breast tomosynthesis, handheld US, automated breast US, and MRI in non-high-risk patients with dense breasts and mammography negative for cancer were reviewed. Data metrics and risk of bias were assessed. Random-effects meta-analysis and two-sided metaregression analyses comparing each imaging modality metrics were performed (PROSPERO; CRD42018080402). Results Twenty-two studies reporting 261 233 screened patients were included. Of 132 166 screened patients with dense breast and mammography negative for cancer who met inclusion criteria, a total of 541 cancers missed at mammography were detected with these supplemental modalities. Metaregression models showed that MRI was superior to other supplemental modalities in CDR (incremental CDR, 1.52 per 1000 screenings; 95% CI: 0.74, 2.33; P < .001), including invasive CDR (invasive CDR, 1.31 per 1000 screenings; 95% CI: 0.57, 2.06; P < .001), and in situ disease (rate of ductal carcinoma in situ, 1.91 per 1000 screenings; 95% CI: 0.10, 3.72; P < .04). No differences in PPV1 and PPV3 were identified. The limited number of studies prevented assessment of interval cancer metrics. Excluding MRI, no statistically significant difference in any metrics were identified among the remaining imaging modalities. Conclusion The pooled data showed that MRI was the best supplemental imaging modality in women at average risk or intermediate risk for breast cancer with dense breasts and mammography negative for cancer. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Hooley and Butler in this issue.
Training doctors’ social skills to break bad news: evaluation of the impact of virtual environment displays on the sense of presence

The way doctors deliver bad news has a significant impact on the therapeutic process. In order to facilitate doctor’s training, we have developed an embodied conversational agent simulating a patient to train doctors to break bad news. In this article, we present an evaluation of the virtual reality training platform comparing the users’ experience depending on the virtual environment displays: a PC desktop, a virtual reality headset, and four wall fully immersive systems. The results of the experience, including both real doctors and naive participants, reveal a significant impact of the environment display on the perception of the user (sense of presence, sense of co-presence, perception of the believability of the virtual patient), showing, moreover, the different perceptions of the participants depending on their level of expertise.
PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Apparent personality and emotion analysis are both central to affective computing.Existing works solve them individually.In this paper we investigate if such high-level affect traits and their relationship can be jointly learned from face images in the wild.To this end, we introduce PersEmoN, an end-to-end trainable and deep Siamese-like network.It consists of two convolutional network branches, one for emotion and the other for apparent personality.Both networks share their bottom feature extraction module and are optimized within a multi-task learning framework.Emotion and personality networks are dedicated to their own annotated dataset.Furthermore, an adversarial-like loss function is employed to promote representation coherence among heterogeneous dataset sources.Based on this, we also explore the emotion-to-apparent-personality relationship.Extensive experiments demonstrate the effectiveness of PersEmoN.
Detecting Hypoglycemia Incidents Reported in Patients’ Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance

Background: Improper dosing of medications such as insulin can cause hypoglycemic episodes, which may lead to severe morbidity or even death. Although secure messaging was designed for exchanging nonurgent messages, patients sometimes report hypoglycemia events through secure messaging. Detecting these patient-reported adverse events may help alert clinical teams and enable early corrective actions to improve patient safety. Objective: We aimed to develop a natural language processing system, called HypoDetect (Hypoglycemia Detector), to automatically identify hypoglycemia incidents reported in patients’ secure messages. Methods: An expert in public health annotated 3000 secure message threads between patients with diabetes and US Department of Veterans Affairs clinical teams as containing patient-reported hypoglycemia incidents or not. A physician independently annotated 100 threads randomly selected from this dataset to determine interannotator agreement. We used this dataset to develop and evaluate HypoDetect. HypoDetect incorporates 3 machine learning algorithms widely used for text classification: linear support vector machines, random forest, and logistic regression. We explored different learning features, including new knowledge-driven features. Because only 114 (3.80%) messages were annotated as positive, we investigated cost-sensitive learning and oversampling methods to mitigate the challenge of imbalanced data. Results: The interannotator agreement was Cohen kappa=.976. Using cross-validation, logistic regression with cost-sensitive learning achieved the best performance (area under the receiver operating characteristic curve=0.954, sensitivity=0.693, specificity 0.974, F1 score=0.590). Cost-sensitive learning and the ensembled synthetic minority oversampling technique improved the sensitivity of the baseline systems substantially (by 0.123 to 0.728 absolute gains). Our results show that a variety of features contributed to the best performance of HypoDetect. Conclusions: Despite the challenge of data imbalance, HypoDetect achieved promising results for the task of detecting hypoglycemia incidents from secure messages. The system has a great potential to facilitate early detection and treatment of hypoglycemia.
Quantum Simulation of Resonant Transitions for Solving the Eigenproblem of an Effective Water Hamiltonian

It is difficult to calculate the energy levels and eigenstates of a large physical system on a classical computer because of the exponentially growing size of the Hilbert space. In this work, we experimentally demonstrate a quantum algorithm which could solve this problem via simulated resonant transitions. Using a four-qubit quantum simulator in which two qubits are used as ancillas for control and measurement, we obtain the energy spectrum of a 2-qubit low-energy effective Hamiltonian of the water molecule. The simulated transitions allow the state of the quantum simulator to transform and access large regions of the Hilbert space, including states that have no overlap with the initial state. Furthermore, we make use of this algorithm to efficiently prepare specific eigenstates on the simulator according to the measured eigenenergies.
Gamification Techniques for Raising Cyber Security Awareness

Due to the prevalence of online services in modern society, such as internet banking and social media, it is important for users to have an understanding of basic security measures in order to keep themselves safe online. However, users often do not know how to make their online interactions secure, which demonstrates an educational need in this area. Gamification has grown in popularity in recent years and has been used to teach people about a range of subjects. This paper presents an exploratory study investigating the use of gamification techniques to educate average users about password security, with the aim of raising overall security awareness. To explore the impact of such techniques, a role-playing quiz application (RPG) was developed for the Android platform to educate users about password security. Results gained from the work highlighted that users enjoyed learning via the use of the password application, and felt they benefitted from the inclusion of gamification techniques. Future work seeks to expand the prototype into a full solution, covering a range of security awareness issues.
Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training

Tasnim Mohiuddin, Shafiq Joty. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Context-Aware Semantic Inpainting

In recent times, image inpainting has witnessed rapid progress due to the generative adversarial networks (GANs) that are able to synthesize realistic contents. However, most existing GAN-based methods for semantic inpainting apply an auto-encoder architecture with a fully connected layer, which cannot accurately maintain spatial information. In addition, the discriminator in existing GANs struggles to comprehend high-level semantics within the image context and yields semantically consistent content. Existing evaluation criteria are biased toward blurry results and cannot well characterize edge preservation and visual authenticity in the inpainting results. In this paper, we propose an improved GAN to overcome the aforementioned limitations. Our proposed GAN-based framework consists of a fully convolutional design for the generator which helps to better preserve spatial structures and a joint loss function with a revised perceptual loss to capture high-level semantics in the context. Furthermore, we also introduce two novel measures to better assess the quality of image inpainting results. The experimental results demonstrate that our method outperforms the state-of-the-art under a wide range of criteria.
A visually realistic grasping system for object manipulation and interaction in virtual reality environments

Interaction in virtual reality (VR) environments (e.g. grasping and manipulating virtual objects) is essential to ensure a pleasant and immersive experience. In this work, we propose a visually realistic, flexible and robust grasping system that enables real-time interactions in virtual environments. Resulting grasps are visually realistic because hand is automatically fitted to the object shape from a position and orientation determined by the user using the VR handheld controllers (e.g. Oculus Touch motion controllers). Our approach is flexible because it can be adapted to different hand meshes (e.g. human or robotic hands) and it is also easily customizable. Moreover, it enables interaction with different objects regardless their geometries. In order to validate our proposal, an exhaustive qualitative and quantitative performance analysis has been carried out. On one hand, qualitative evaluation was used in the assessment of abstract aspects, such as motor control, finger movement realism, and interaction realism. On the other hand, for the quantitative evaluation a novel metric has been proposed to visually analyze the performed grips. Performance analysis results indicate that previous experience with our grasping system is not a prerequisite for an enjoyable, natural and intuitive VR interaction experience.
Two-Stream Region Convolutional 3D Network for Temporal Activity Detection

We address the problem of temporal activity detection in continuous, untrimmed video streams. This is a difficult task that requires extracting meaningful spatio-temporal features to capture activities, accurately localizing the start and end times of each activity. We introduce a new model, Region Convolutional 3D Network (R-C3D), which encodes the video streams using a three-dimensional fully convolutional network, then generates candidate temporal regions containing activities and finally classifies selected regions into specific activities. Computation is saved due to the sharing of convolutional features between the proposal and the classification pipelines. We further improve the detection performance by efficiently integrating an optical flow based motion stream with the original RGB stream. The two-stream network is jointly optimized by fusing the flow and RGB feature maps at different levels. Additionally, the training stage incorporates an online hard example mining strategy to address the extreme foreground-background imbalance typically observed in any detection pipeline. Instead of heuristically sampling the candidate segments for the final activity classification stage, we rank them according to their performance and only select the worst performers to update the model. This improves the model without heavy hyper-parameter tuning. Extensive experiments on three benchmark datasets are carried out to show superior performance over existing temporal activity detection methods. Our model achieves state-of-the-art results on the THUMOS'14 and Charades datasets. We further demonstrate that our model is a general temporal activity detection framework that does not rely on assumptions about particular dataset properties by evaluating our approach on the ActivityNet dataset.
Globally-Aware Multiple Instance Classifier for Breast Cancer Screening

Deep learning models designed for visual classification tasks on natural images have become prevalent in medical image analysis. However, medical images differ from typical natural images in many ways, such as significantly higher resolutions and smaller regions of interest. Moreover, both the global structure and local details play important roles in medical image analysis tasks. To address these unique properties of medical images, we propose a neural network that is able to classify breast cancer lesions utilizing information from both a global saliency map and multiple local patches. The proposed model outperforms the ResNet-based baseline and achieves radiologist-level performance in the interpretation of screening mammography. Although our model is trained only with image-level labels, it is able to generate pixel-level saliency maps that provide localization of possible malignant findings.
Deep Neural Architecture Search with Deep Graph Bayesian Optimization

Image recognition aims to identify objects, places, people, or other targeted items in a given image, and has a wide range of social applications such as natural disasters recognition, plant disease detection, and traffic jam detection. Currently state-of-the-art methods of image recognition are based on deep learning and remain a common pattern in designing and using convolutional neural networks (CNNs). However, designing CNNs is extremely time intensive and requires an expert. Neural architecture search (NAS) can solve this problem by automatically identifying architectures of CNNs that are superior to hand-designed ones. Recently BO has been applied to neural architecture search and shows better performance than pure evolutionary strategies. All these methods adopt Gaussian processes (GPs) as surrogate function, with the handcraft similarity metrics as input. In this work, we propose a Bayesian graph neural network as a new surrogate, which can automatically extract features from deep neural architectures, and use such learned features to fit and characterize black-box objectives and their uncertainty. Based on the new surrogate, we then develop a graph Bayesian optimization framework to address the challenging task of deep neural architecture search. Experiment results show our method significantly outperforms the comparative methods on benchmark tasks.
Trajectory-based recognition of dynamic Persian sign language using hidden Markov model

Sign Language Recognition (SLR) is an important step in facilitating the communication among deaf people and the rest of society. Existing Persian sign language recognition systems are mainly restricted to static signs which are not very useful in everyday communications. In this study, a dynamic Persian sign language recognition system is presented. A collection of 1200 videos were captured from 12 individuals performing 20 dynamic signs with a simple white glove. The trajectory of the hands, along with hand shape information were extracted from each video using a simple region-growing technique. These time-varying trajectories were then modeled using Hidden Markov Model (HMM) with Gaussian probability density functions as observations. The performance of the system was evaluated in different experimental strategies. Signer-independent and signer-dependent experiments were performed on the proposed system and the average accuracy of 97.48% was obtained. The experimental results demonstrated that the performance of the system is independent of the subject and it can also perform excellently even with a limited number of training data.
SITUP: Scale Invariant Tracking Using Average Peak-to-Correlation Energy

Robust and accurate scale estimation of a target object is a challenging task in visual object tracking. Most existing tracking methods cannot accommodate large scale variation in complex image sequences and thus result in inferior performance. In this paper, we propose to incorporate a novel criterion called the average peak-to-correlation energy into the multi-resolution translation filter framework to obtain robust and accurate scale estimation. The resulting system is named SITUP: Scale Invariant Tracking using Average Peak-to-Correlation Energy. SITUP effectively tackles the problem of fixed template size in standard discriminative correlation filter based trackers. Extensive empirical evaluation on the publicly available tracking benchmark datasets demonstrates that the proposed scale searching framework meets the demands of scale variation challenges effectively while providing superior performance over other scale adaptive variants of standard discriminative correlation filter based trackers. Also, SITUP obtains favorable performance compared to state-of-the-art trackers for various scenarios while operating in real-time on a single CPU.
Genome assembly using quantum and quantum-inspired annealing

Recent advances in DNA sequencing open prospects to make whole-genome analysis rapid and reliable, which is promising for various applications including personalized medicine. However, existing techniques for de novo genome assembly, which is used for the analysis of genomic rearrangements, chromosome phasing, and reconstructing genomes without a reference, require solving tasks of high computational complexity. Here we demonstrate a method for solving genome assembly tasks with the use of quantum and quantum-inspired optimization techniques. Within this method, we present experimental results on genome assembly using quantum annealers both for simulated data and the [Formula: see text]X 174 bacteriophage. Our results pave a way for a significant increase in the efficiency of solving bioinformatics problems with the use of quantum computing technologies and, in particular, quantum annealing might be an effective method. We expect that the new generation of quantum annealing devices would outperform existing techniques for de novo genome assembly. To the best of our knowledge, this is the first experimental study of de novo genome assembly problems both for real and synthetic data on quantum annealing devices and quantum-inspired techniques.
Tightening the tripartite quantum-memory-assisted entropic uncertainty relation

The uncertainty principle determines the distinction between the classical and the quantum world. This principle states that it is not possible to measure two incompatible observables with the desired accuracy simultaneously. In quantum information theory, the Shannon entropy is used as an appropriate measure to express the uncertainty relation. Improving the bound of the entropic uncertainty relation is of great importance. The bound can be varied by considering an extra quantum system as the quantum memory which is correlated with the measured quantum system. One can extend the bipartite quantum-memory-assisted entropic uncertainty relation to the tripartite one in which the memory is split into two parts. Here, a lower bound is obtained for the tripartite quantum-memory-assisted entropic uncertainty relation. This lower bound has two extra terms compared with the lower bound in Renes and Boileau [J. M. Renes and J.-C. Boileau, Phys. Rev. Lett. 110, 020402 (2013)] which depends on the conditional von Neumann entropy, the Holevo quantity, and the mutual information. It is shown that the bound is tighter than other bounds derived earlier. It also leads to a lower bound for the quantum secret key rate. In addition, it is applied to obtain the states for which both the strong subadditivity and the Koashi-Winter inequalities turn into equalities.
A Cross-Task Analysis of Text Span Representations

Many natural language processing (NLP) tasks involve reasoning with textual spans, including question answering, entity recognition, and coreference resolution. While extensive research has focused on functional architectures for representing words and sentences, there is less work on representing arbitrary spans of text within sentences. In this paper, we conduct a comprehensive empirical evaluation of six span representation methods using eight pretrained language representation models across six tasks, including two tasks that we introduce. We find that, although some simple span representations are fairly reliable across tasks, in general the optimal span representation varies by task, and can also vary within different facets of individual tasks. We also find that the choice of span representation has a bigger impact with a fixed pretrained encoder than with a fine-tuned encoder.