Reviewerly/exHarmony huggingface dataset:
Quantum jump metrology in a two-cavity network

Quantum metrology enhances measurement precision by utilising the properties of quantum physics. In interferometry, this is typically achieved by evolving highly-entangled quantum states before performing single-shot measurements to reveal information about an unknown parameter. While this is often the optimum approach, implementation with all but the smallest states is still extremely challenging. An alternative approach is quantum jump metrology [L. A. Clark et al., Phys. Rev. A 99, 022102 (2019)] which deduces information by continuously monitoring an open quantum system, while inducing phase-dependent temporal correlations with the help of quantum feedback. Taking this approach here, we analyse measurements of a relative phase in an optical network of two cavities with quantum feedback in the form of laser pulses. It is shown that the proposed approach can exceed the standard quantum limit without the need for complex quantum states while being scalable and more practical than previous related schemes.
Probabilistic Autoencoder

Principal Component Analysis (PCA) minimizes the reconstruction error given a class of linear models of fixed component dimensionality. Probabilistic PCA adds a probabilistic structure by learning the probability distribution of the PCA latent space weights, thus creating a generative model. Autoencoders (AE) minimize the reconstruction error in a class of nonlinear models of fixed latent space dimensionality and outperform PCA at fixed dimensionality. Here, we introduce the Probabilistic Autoencoder (PAE) that learns the probability distribution of the AE latent space weights using a normalizing flow (NF). The PAE is fast and easy to train and achieves small reconstruction errors, high sample quality, and good performance in downstream tasks. We compare the PAE to Variational AE (VAE), showing that the PAE trains faster, reaches a lower reconstruction error, and produces good sample quality without requiring special tuning parameters or training procedures. We further demonstrate that the PAE is a powerful model for performing the downstream tasks of probabilistic image reconstruction in the context of Bayesian inference of inverse problems for inpainting and denoising applications. Finally, we identify latent space density from NF as a promising outlier detection metric.
Real-time Online Video Detection with Temporal Smoothing Transformers

Streaming video recognition reasons about objects and their actions in every frame of a video. A good streaming recognition model captures both long-term dynamics and short-term changes of video. Unfortunately, in most existing methods, the computational complexity grows linearly or quadratically with the length of the considered dynamics. This issue is particularly pronounced in transformer-based architectures. To address this issue, we reformulate the cross-attention in a video transformer through the lens of kernel and apply two kinds of temporal smoothing kernel: A box kernel or a Laplace kernel. The resulting streaming attention reuses much of the computation from frame to frame, and only requires a constant time update each frame. Based on this idea, we build TeSTra, a Temporal Smoothing Transformer, that takes in arbitrarily long inputs with constant caching and computing overhead. Specifically, it runs $6\times$ faster than equivalent sliding-window based transformers with 2,048 frames in a streaming setting. Furthermore, thanks to the increased temporal span, TeSTra achieves state-of-the-art results on THUMOS'14 and EPIC-Kitchen-100, two standard online action detection and action anticipation datasets. A real-time version of TeSTra outperforms all but one prior approaches on the THUMOS'14 dataset.
Generating detailed saliency maps using model-agnostic methods

The emerging field of Explainable Artificial Intelligence focuses on researching methods of explaining the decision making processes of complex machine learning models. In the field of explainability for Computer Vision, explanations are provided as saliency maps, which visualize the importance of individual pixels of the input w.r.t. the model's prediction. In this work we focus on a perturbation-based, model-agnostic explainability method called RISE, elaborate on observed shortcomings of its grid-based approach and propose two modifications: replacement of square occlusions with convex polygonal occlusions based on cells of a Voronoi mesh and addition of an informativeness guarantee to the occlusion mask generator. These modifications, collectively called VRISE (Voronoi-RISE), are meant to, respectively, improve the accuracy of maps generated using large occlusions and accelerate convergence of saliency maps in cases where sampling density is either very low or very high. We perform a quantitative comparison of accuracy of saliency maps produced by VRISE and RISE on the validation split of ILSVRC2012, using a saliency-guided content insertion/deletion metric and a localization metric based on bounding boxes. Additionally, we explore the space of configurable occlusion pattern parameters to better understand their influence on saliency maps produced by RISE and VRISE. We also describe and demonstrate two effects observed over the course of experimentation, arising from the random sampling approach of RISE: "feature slicing" and "saliency misattribution". Our results show that convex polygonal occlusions yield more accurate maps for coarse occlusion meshes and multi-object images, but improvement is not guaranteed in other cases. The informativeness guarantee is shown to increase the convergence rate without incurring a significant computational overhead.
Cache-Oblivious Representation of B-Tree Structures

We present a data structure CORoBTS for storing a search tree with all leaves at the same depth and vertices of arity between chosen constants $a$ and $b$ in a cache-oblivious way. It provides operations for inserting an $a$-ary subtree and removing a subtree; both have an amortized I/O complexity $\mathcal{O}(S\cdot(\log^2 N)/ B + \log_B N \cdot \log\log S + 1)$ and amortized time complexity $\mathcal{O}(S\cdot\log^2 N)$, where $S$ is the size of the subtree and $N$ size of the whole stored tree. The tree allows searching with an optimal I/O complexity $\mathcal{O}(\log_B{N})$ and is stored in a linear space. We use the data structure as a top space-time tree in the cache-oblivious partially persistent array proposed by Davoodi et al. [DFI\"O14]. The space complexity of the persistent array is then improved from $\mathcal{O}(U^{\log_2 3} + V \log U)$ to $\mathcal{O}(U + V \log U)$, where $U$ is the maximal size of the array and $V$ is the number of versions. The data locality and I/O complexity of both present and persistent reads are kept unchanged; I/O complexity of writes is worsened by a polylogarithmic factor.
Table Detection in the Wild: A Novel Diverse Table Detection Dataset and Method

Recent deep learning approaches in table detection achieved outstanding performance and proved to be effective in identifying document layouts. Currently, available table detection benchmarks have many limitations, including the lack of samples diversity, simple table structure, the lack of training cases, and samples quality. In this paper, we introduce a diverse large-scale dataset for table detection with more than seven thousand samples containing a wide variety of table structures collected from many diverse sources. In addition to that, we also present baseline results using a convolutional neural network-based method to detect table structure in documents. Experimental results show the superiority of applying convolutional deep learning methods over classical computer vision-based methods. The introduction of this diverse table detection dataset will enable the community to develop high throughput deep learning methods for understanding document layout and tabular data processing. Dataset is available at: 1. https://www.kaggle.com/datasets/mrinalim/stdw-dataset 2. https://huggingface.co/datasets/n3011/STDW
DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using Attention-Based Generative Adverserial Network

Human motion prediction is a fundamental part of many human-robot applications. Despite the recent progress in human motion prediction, most studies simplify the problem by predicting the human motion relative to a fixed joint and/or only limit their model to predict one possible future motion. While due to the complex nature of human motion, a single output cannot reflect all the possible actions one can do. Also, for any robotics application, we need the full human motion including the user trajectory not a 3d pose relative to the hip joint. In this paper, we try to address these two issues by proposing a transformer-based generative model for forecasting multiple diverse human motions. Our model generates \textit{N} future possible motion by querying a history of human motion. Our model first predicts the pose of the body relative to the hip joint. Then the \textit{Hip Prediction Module} predicts the trajectory of the hip movement for each predicted pose frame. To emphasize on the diverse future motions we introduce a similarity loss that penalizes the pairwise sample distance. We show that our system outperforms the state-of-the-art in human motion prediction while it can predict diverse multi-motion future trajectories with hip movements
Semantic-based Pre-training for Dialogue Understanding

Pre-trained language models have made great progress on dialogue tasks. However, these models are typically trained on surface dialogue text, thus are proven to be weak in understanding the main semantic meaning of a dialogue context. We investigate Abstract Meaning Representation (AMR) as explicit semantic knowledge for pre-training models to capture the core semantic information in dialogues during pre-training. In particular, we propose a semantic-based pre-training framework that extends the standard pre-training framework (Devlin et al., 2019) by three tasks for learning 1) core semantic units, 2) semantic relations and 3) the overall semantic representation according to AMR graphs. Experiments on the understanding of both chit-chats and task-oriented dialogues show the superiority of our model. To our knowledge, we are the first to leverage a deep semantic representation for dialogue pre-training.
Comparing Middle School Student Learning Outcomes from In-Person, Online, and Hybrid Instructions: A Consideration of Learning Engagement during a Pandemic

Learning engagement varies in different modes of instructions. With such consideration, the authors of this article conducted comparative analyses on student learning outcomes measured by their final mean grade point averages (GPAs) during three timeperiods (before, at the onset of, and during pandemic) when inperson, online, and hybrid instructions were delivered. The records of 600 middle school students in a school district from a western state, across three academic years, were used as the sample (2018-2019 before pandemic when they were in sixth grade with in-person instruction, 2019-2020 at the onset of the COVID-19 pandemic in seventh grade with online instruction, and 2020-2021 during pandemic in eighth grade with both online and hybrid instructions). Results showed that no difference was found (from paired t-test) in student final GPAs when they transferred from in-person instruction before pandemic to online instruction at the onset of pandemic, but differences were found (a) between in-person instruction and hybrid instruction (from one-way repeated ANOVA), and (b) between online instruction and hybrid instruction (from both one-way repeated ANOVA and an independent samples t-test) at and during pandemic. Recommendations for practice and future research focusing on students’ needs were presented.
How to Adapt Pre-trained Vision-and-Language Models to a Text-only Input?

Current language models have been criticised for learning language from text alone without connection between words and their meaning. Consequently, multimodal training has been proposed as a way for creating models with better language understanding by providing the lacking connection. We focus on pre-trained multimodal vision-and-language (VL) models for which there already are some results on their language understanding capabilities. An unresolved issue with evaluating the linguistic skills of these models, however, is that there is no established method for adapting them to text-only input without out-of-distribution uncertainty. To find the best approach, we investigate and compare seven possible methods for adapting three different pre-trained VL models to text-only input. Our evaluations on both GLUE and Visual Property Norms (VPN) show that care should be put into adapting VL models to zero-shot text-only tasks, while the models are less sensitive to how we adapt them to non-zero-shot tasks. We also find that the adaptation methods perform differently for different models and that unimodal model counterparts perform on par with the VL models regardless of adaptation, indicating that current VL models do not necessarily gain better language understanding from their multimodal training.
How to Create Universes with Internal Flux

String compactifications typically require fluxes, for example in order to stabilise moduli. Such fluxes, when they thread internal dimensions, are topological in nature and take on quantised values. This poses the puzzle as to how they could arise in the early universe, as they cannot be turned on incrementally. Working with string inspired models in $6$ and $8$ dimensions, we show that there exist no-boundary solutions in which internal fluxes are present from the creation of the universe onwards. The no-boundary proposal can thus explain the origin of fluxes in a Kaluza-Klein context. In fact, it acts as a selection principle since no-boundary solutions are only found to exist when the fluxes have the right magnitude to lead to an effective potential that is positive and flat enough for accelerated expansion. Within the range of selected fluxes, the no-boundary wave function assigns higher probability to smaller values of flux. Our models illustrate how cosmology can act as a filter on a landscape of possible higher-dimensional solutions.
Will It Blend? Mixing Training Paradigms &amp; Prompting for Argument Quality Prediction

This paper describes our contributions to the Shared Task of the 9th Workshop on Argument Mining (2022). Our approach uses Large Language Models for the task of Argument Quality Prediction. We perform prompt engineering using GPT-3, and also investigate the training paradigms multi-task learning, contrastive learning, and intermediate-task training. We find that a mixed prediction setup outperforms single models. Prompting GPT-3 works best for predicting argument validity, and argument novelty is best estimated by a model trained using all three training paradigms.
An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey

The reinforcement learning (RL) research area is very active, with an important number of new contributions; especially considering the emergent field of deep RL (DRL). However a number of scientific and technical challenges still need to be resolved, amongst which we can mention the ability to abstract actions or the difficulty to explore the environment in sparse-reward settings which can be addressed by intrinsic motivation (IM). We propose to survey these research works through a new taxonomy based on information theory: we computationally revisit the notions of surprise, novelty and skill learning. This allows us to identify advantages and disadvantages of methods and exhibit current outlooks of research. Our analysis suggests that novelty and surprise can assist the building of a hierarchy of transferable skills that further abstracts the environment and makes the exploration process more robust.
A model-agnostic approach for generating Saliency Maps to explain inferred decisions of Deep Learning Models

The widespread use of black-box AI models has raised the need for algorithms and methods that explain the decisions made by these models. In recent years, the AI research community is increasingly interested in models' explainability since black-box models take over more and more complicated and challenging tasks. Explainability becomes critical considering the dominance of deep learning techniques for a wide range of applications, including but not limited to computer vision. In the direction of understanding the inference process of deep learning models, many methods that provide human comprehensible evidence for the decisions of AI models have been developed, with the vast majority relying their operation on having access to the internal architecture and parameters of these models (e.g., the weights of neural networks). We propose a model-agnostic method for generating saliency maps that has access only to the output of the model and does not require additional information such as gradients. We use Differential Evolution (DE) to identify which image pixels are the most influential in a model's decision-making process and produce class activation maps (CAMs) whose quality is comparable to the quality of CAMs created with model-specific algorithms. DE-CAM achieves good performance without requiring access to the internal details of the model's architecture at the cost of more computational complexity.
CFVS: Coarse-to-Fine Visual Servoing for 6-DoF Object-Agnostic Peg-In-Hole Assembly

Robotic peg-in-hole assembly remains a challenging task due to its high accuracy demand. Previous work tends to simplify the problem by restricting the degree of freedom of the end-effector, or limiting the distance between the target and the initial pose position, which prevents them from being deployed in real-world manufacturing. Thus, we present a Coarse-to-Fine Visual Servoing (CFVS) peg-in-hole method, achieving 6-DoF end-effector motion control based on 3D visual feedback. CFVS can handle arbitrary tilt angles and large initial alignment errors through a fast pose estimation before refinement. Furthermore, by introducing a confidence map to ignore the irrelevant contour of objects, CFVS is robust against noise and can deal with various targets beyond training data. Extensive experiments show CFVS outperforms state-of-the-art methods and obtains 100%, 91%, and 82% average success rates in 3-DoF, 4-DoF, and 6-DoF peg-in-hole, respectively.
MAGES 4.0: Accelerating the world's transition to VR training and democratizing the authoring of the medical metaverse

In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked participants can also collaborate using different virtual/augmented reality as well as mobile and desktop devices, in the same metaverse world. With MAGES we propose an upgrade to the outdated 150-year-old master-apprentice medical training model. Our platform incorporates, in a nutsell, the following novelties: a) 5G edge-cloud remote rendering and physics dissection layer, b) realistic real-time simulation of organic tissues as soft-bodies under 10ms, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or debrief the training simulation from any perspective.
Belief Revision based Caption Re-ranker with Visual Semantic Information

In this work, we focus on improving the captions generated by image-caption generation systems. We propose a novel re-ranking approach that leverages visual-semantic measures to identify the ideal caption that maximally captures the visual information in the image. Our re-ranker utilizes the Belief Revision framework (Blok et al., 2003) to calibrate the original likelihood of the top-n captions by explicitly exploiting the semantic relatedness between the depicted caption and the visual context. Our experiments demonstrate the utility of our approach, where we observe that our re-ranker can enhance the performance of a typical image-captioning system without the necessity of any additional training or fine-tuning.
Breaking the Speed Limit for Perfect Quantum State Transfer

We describe a protocol for perfectly transferring a quantum state from one party to another under the dynamics of a fixed, engineered Hamiltonian. Our protocol combines the concepts of fractional revival, dual rail encoding, and a rare glimpse of the anti-Zeno effect. Remarkably, the transfer happens faster than the speed limit for perfect quantum state transfer [1, 2].
Multi-channel Nuclear Norm Minus Frobenius Norm Minimization for Color Image Denoising

Color image denoising is frequently encountered in various image processing and computer vision tasks. One traditional strategy is to convert the RGB image to a less correlated color space and denoise each channel of the new space separately. However, such a strategy can not fully exploit the correlated information between channels and is inadequate to obtain satisfactory results. To address this issue, this paper proposes a new multi-channel optimization model for color image denoising under the nuclear norm minus Frobenius norm minimization framework. Specifically, based on the block-matching, the color image is decomposed into overlapping RGB patches. For each patch, we stack its similar neighbors to form the corresponding patch matrix. The proposed model is performed on the patch matrix to recover its noise-free version. During the recovery process, a) a weight matrix is introduced to fully utilize the noise difference between channels; b) the singular values are shrunk adaptively without additionally assigning weights. With them, the proposed model can achieve promising results while keeping simplicity. To solve the proposed model, an accurate and effective algorithm is built based on the alternating direction method of multipliers framework. The solution of each updating step can be analytically expressed in closed-from. Rigorous theoretical analysis proves the solution sequences generated by the proposed algorithm converge to their respective stationary points. Experimental results on both synthetic and real noise datasets demonstrate the proposed model outperforms state-of-the-art models.
Blockchain based secure energy marketplace scheme to motivate P2P microgrids

In the past years trend of microgrids is increasing very fast to reduce peak-hour costs. However, in these systems, third parties are still involved in selling surplus energy. This results in increased cost of energy and there are many operational and security barriers in such systems. These issues can be solved by the decentralized distributed system of microgrids where a consumer can locally sell their surplus energy to another consumer. To deploy such a system, one must consider security barriers for the transaction of energy. This paper proposes a solution to these problems by devising a scheme as a marketplace where users interact with each other to buy and sell energy at better rates and get energy-generating resources on lease so that users do not have to worry about capital investment. Agreement between owner of resources and consumer is recorded on blockchain based smart contracts. In this paper, a survey is performed for existing well known, decentralized energy solutions. This paper also proposes an extra layer of security to leverage a shielded execution environment so that information of energy generated, utilized, and shared cannot be changed by consumers and third parties even if the system is compromised.