Reviewerly/exHarmony huggingface dataset:
Reverse Stable Diffusion: What prompt was used to generate this image?

Text-to-image diffusion models such as Stable Diffusion have recently attracted the interest of many researchers, and inverting the diffusion process can play an important role in better understanding the generative process and how to engineer prompts in order to obtain the desired images. To this end, we introduce the new task of predicting the text prompt given an image generated by a generative diffusion model. We combine a series of white-box and black-box models (with and without access to the weights of the diffusion network) to deal with the proposed task. We propose a novel learning framework comprising of a joint prompt regression and multi-label vocabulary classification objective that generates improved prompts. To further improve our method, we employ a curriculum learning procedure that promotes the learning of image-prompt pairs with lower labeling noise (i.e. that are better aligned), and an unsupervised domain-adaptive kernel learning method that uses the similarities between samples in the source and target domains as extra features. We conduct experiments on the DiffusionDB data set, predicting text prompts from images generated by Stable Diffusion. Our novel learning framework produces excellent results on the aforementioned task, yielding the highest gains when applied on the white-box model. In addition, we make an interesting discovery: training a diffusion model on the prompt generation task can make the model generate images that are much better aligned with the input prompts, when the model is directly reused for text-to-image generation.
Peer Surveillance in Online Communities

Online communities are not safe spaces for user privacy. Even though existing research focuses on creating and improving various content moderation strategies and privacy preserving technologies, platforms hosting online communities support features allowing users to surveil one another--leading to harassment, personal data breaches, and offline harm. To tackle this problem, we introduce a new, work-in-progress framework for analyzing data privacy within vulnerable, identity-based online communities. Where current SOUPS papers study surveillance and longitudinal user data as two distinct challenges to user privacy, more work needs to be done in exploring the sites where surveillance and historical user data assemble. By synthesizing over 40 years of developments in the analysis of surveillance, we derive properties of online communities that enable the abuse of user data by fellow community members and suggest key steps to improving security for vulnerable users. Deploying this new framework on new and existing platforms will ensure that online communities are privacy-conscious and designed more inclusively.
Analysing QBER and secure key rate under various losses for satellite based free space QKD

Quantum Key Distribution is a key distribution method that uses the qubits to safely distribute one-time use encryption keys between two or more authorised participants in a way that ensures the identification of any eavesdropper. In this paper, we have done a comparison between the BB84 and B92 protocols and BBM92 and E91 entanglement based protocols for satellite based uplink and downlink in low Earth orbit. The expressions for the quantum bit error rate and the keyrate are given for all four protocols. The results indicate that, when compared to the B92 protocol, the BB84 protocol guarantees the distribution of a higher secure keyrate for a specific distance. Similarly, it is observed that BBM92 ensures higher keyrate in comparison with E91 protocol.
A Probabilistic Approach to Self-Supervised Learning using Cyclical Stochastic Gradient MCMC

In this paper we present a practical Bayesian self-supervised learning method with Cyclical Stochastic Gradient Hamiltonian Monte Carlo (cSGHMC). Within this framework, we place a prior over the parameters of a self-supervised learning model and use cSGHMC to approximate the high dimensional and multimodal posterior distribution over the embeddings. By exploring an expressive posterior over the embeddings, Bayesian self-supervised learning produces interpretable and diverse representations. Marginalizing over these representations yields a significant gain in performance, calibration and out-of-distribution detection on a variety of downstream classification tasks. We provide experimental results on multiple classification tasks on four challenging datasets. Moreover, we demonstrate the effectiveness of the proposed method in out-of-distribution detection using the SVHN and CIFAR-10 datasets.
DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales

ChatGPT-like models have revolutionized various applications in artificial intelligence, from summarization and coding to translation, matching or even surpassing human performance. However, the current landscape lacks an accessible, efficient, and cost-effective end-to-end RLHF (Reinforcement Learning with Human Feedback) training pipeline for these powerful models, particularly when training at the scale of billions of parameters. This paper introduces DeepSpeed-Chat, a novel system that democratizes RLHF training, making it accessible to the AI community. DeepSpeed-Chat offers three key capabilities: an easy-to-use training and inference experience for ChatGPT-like models, a DeepSpeed-RLHF pipeline that replicates the training pipeline from InstructGPT, and a robust DeepSpeed-RLHF system that combines various optimizations for training and inference in a unified way. The system delivers unparalleled efficiency and scalability, enabling training of models with hundreds of billions of parameters in record time and at a fraction of the cost. With this development, DeepSpeed-Chat paves the way for broader access to advanced RLHF training, even for data scientists with limited resources, thereby fostering innovation and further development in the field of AI.
Efficient neural supersampling on a novel gaming dataset

Real-time rendering for video games has become increasingly challenging due to the need for higher resolutions, framerates and photorealism. Supersampling has emerged as an effective solution to address this challenge. Our work introduces a novel neural algorithm for supersampling rendered content that is 4 times more efficient than existing methods while maintaining the same level of accuracy. Additionally, we introduce a new dataset which provides auxiliary modalities such as motion vectors and depth generated using graphics rendering features like viewport jittering and mipmap biasing at different resolutions. We believe that this dataset fills a gap in the current dataset landscape and can serve as a valuable resource to help measure progress in the field and advance the state-of-the-art in super-resolution techniques for gaming content.
Reasoning in Large Language Models Through Symbolic Math Word Problems

Large language models (LLMs) have revolutionized NLP by solving downstream tasks with little to no labeled data. Despite their versatile abilities, the larger question of their ability to reason remains ill-understood. This paper addresses reasoning in math word problems (MWPs) by studying symbolic versions of the numeric problems, since a symbolic expression is a "concise explanation" of the numeric answer. We create and use a symbolic version of the SVAMP dataset and find that GPT-3's davinci-002 model also has good zero-shot accuracy on symbolic MWPs. To evaluate the faithfulness of the model's reasoning, we go beyond accuracy and additionally evaluate the alignment between the final answer and the outputted reasoning, which correspond to numeric and symbolic answers respectively for MWPs. We explore a self-prompting approach to encourage the symbolic reasoning to align with the numeric answer, thus equipping the LLM with the ability to provide a concise and verifiable reasoning and making it more interpretable. Surprisingly, self-prompting also improves the symbolic accuracy to be higher than both the numeric and symbolic accuracies, thus providing an ensembling effect. The SVAMP_Sym dataset will be released for future research on symbolic math problems.
Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives

Deep learning has demonstrated remarkable performance across various tasks in medical imaging. However, these approaches primarily focus on supervised learning, assuming that the training and testing data are drawn from the same distribution. Unfortunately, this assumption may not always hold true in practice. To address these issues, unsupervised domain adaptation (UDA) techniques have been developed to transfer knowledge from a labeled domain to a related but unlabeled domain. In recent years, significant advancements have been made in UDA, resulting in a wide range of methodologies, including feature alignment, image translation, self-supervision, and disentangled representation methods, among others. In this paper, we provide a comprehensive literature review of recent deep UDA approaches in medical imaging from a technical perspective. Specifically, we categorize current UDA research in medical imaging into six groups and further divide them into finer subcategories based on the different tasks they perform. We also discuss the respective datasets used in the studies to assess the divergence between the different domains. Finally, we discuss emerging areas and provide insights and discussions on future research directions to conclude this survey.
Learning Spatial Distribution of Long-Term Trackers Scores

Long-Term tracking is a hot topic in Computer Vision. In this context, competitive models are presented every year, showing a constant growth rate in performances, mainly measured in standardized protocols as Visual Object Tracking (VOT) and Object Tracking Benchmark (OTB). Fusion-trackers strategy has been applied over last few years for overcoming the known re-detection problem, turning out to be an important breakthrough. Following this approach, this work aims to generalize the fusion concept to an arbitrary number of trackers used as baseline trackers in the pipeline, leveraging a learning phase to better understand how outcomes correlate with each other, even when no target is present. A model and data independence conjecture will be evidenced in the manuscript, yielding a recall of 0.738 on LTB-50 dataset when learning from VOT-LT2022, and 0.619 by reversing the two datasets. In both cases, results are strongly competitive with state-of-the-art and recall turns out to be the first on the podium.
An Adaptable Approach for Successful SIEM Adoption in Companies

In corporations around the world, the topic of cybersecurity and information security is becoming increasingly important as the number of cyberattacks on themselves continues to grow. Nowadays, it is no longer just a matter of protecting against cyberattacks, but rather of detecting such attacks at an early stage and responding accordingly. There is currently no generic methodological approach for the implementation of Security Information and Event Management (SIEM) systems that takes academic aspects into account and can be applied independently of the product or developers of the systems. Applying Hevner's design science research approach, the goal of this paper is to develop a holistic procedure model for implementing respective SIEM systems in corporations. According to the study during the validation phase, the procedure model was verified to be applicable. As desire for future research, the procedure model should be applied in various implementation projects in different enterprises to analyze its applicability and completeness.
Quantum Reservoir Probing of Quantum Information Scrambling

Quantum information scrambling is the dynamical process where initially localized quantum information spreads throughout an entire system. Here, by extending the idea of quantum reservoir computing, we propose the quantum reservoir probing (QRP) to diagnose quantum information scrambling. As a paradigmatic example, we investigate scrambling in a one-dimensional quantum Ising chain. We show that the QRP successfully distinguishes between quasiparticle-mediated information propagation in a free fermion system and correlation-mediated scrambling in a quantum chaotic system. Furthermore, via the scanning of read-out operators, the QRP can elucidate the dynamical distribution of quantum information in the Hilbert space, a distinct advantage over the conventional methods with out-of-time-order correlators and tripartite mutual information.
Context-Aware Talking-Head Video Editing

Talking-head video editing aims to efficiently insert, delete, and substitute the word of a pre-recorded video through a text transcript editor. The key challenge for this task is obtaining an editing model that generates new talking-head video clips which simultaneously have accurate lip synchronization and motion smoothness. Previous approaches, including 3DMM-based (3D Morphable Model) methods and NeRF-based (Neural Radiance Field) methods, are sub-optimal in that they either require minutes of source videos and days of training time or lack the disentangled control of verbal (e.g., lip motion) and non-verbal (e.g., head pose and expression) representations for video clip insertion. In this work, we fully utilize the video context to design a novel framework for talking-head video editing, which achieves efficiency, disentangled motion control, and sequential smoothness. Specifically, we decompose this framework to motion prediction and motion-conditioned rendering: (1) We first design an animation prediction module that efficiently obtains smooth and lip-sync motion sequences conditioned on the driven speech. This module adopts a non-autoregressive network to obtain context prior and improve the prediction efficiency, and it learns a speech-animation mapping prior with better generalization to novel speech from a multi-identity video dataset. (2) We then introduce a neural rendering module to synthesize the photo-realistic and full-head video frames given the predicted motion sequence. This module adopts a pre-trained head topology and uses only few frames for efficient fine-tuning to obtain a person-specific rendering model. Extensive experiments demonstrate that our method efficiently achieves smoother editing results with higher image quality and lip accuracy using less data than previous methods.
Multi-time quantum process tomography of a superconducting qubit

Non-Markovian noise poses a formidable challenge to the scalability of quantum devices, being both ubiquitous in current quantum hardware and notoriously difficult to characterise. This challenge arises from the need for a full reconstruction of a multi-time process, a task that has proven elusive in previous efforts. In this work, we achieve the milestone of complete tomographic characterisation of a multi-time quantum process on a superconducting qubit by employing sequential measure-and-prepare operations with an experimentally motivated post-processing technique, utilising both in-house and cloud-based superconducting quantum processors. Employing the process matrix formalism, we reveal intricate landscapes of non-Markovian noise and provide evidence that components of the noise originate from quantum sources. Our findings and techniques have significant implications for advancing error-mitigation strategies and enhancing the scalability of quantum devices.
Reachability, Coolability, and Stabilizability of Open Markovian Quantum Systems with Fast Unitary Control

Open Markovian quantum systems with fast and full Hamiltonian control can be reduced to an equivalent control system on the standard simplex modelling the dynamics of the eigenvalues of the density matrix describing the quantum state. We explore this reduced control system for answering questions on reachability and stabilizability with immediate applications to the cooling of Markovian quantum systems. We show that for certain tasks of interest, the control Hamiltonian can be chosen time-independent. -- The reduction picture is an example of dissipative interconversion between equivalence classes of states, where the classes are induced by fast controls.
MonoNext: A 3D Monocular Object Detection with ConvNext

Autonomous driving perception tasks rely heavily on cameras as the primary sensor for Object Detection, Semantic Segmentation, Instance Segmentation, and Object Tracking. However, RGB images captured by cameras lack depth information, which poses a significant challenge in 3D detection tasks. To supplement this missing data, mapping sensors such as LIDAR and RADAR are used for accurate 3D Object Detection. Despite their significant accuracy, the multi-sensor models are expensive and require a high computational demand. In contrast, Monocular 3D Object Detection models are becoming increasingly popular, offering a faster, cheaper, and easier-to-implement solution for 3D detections. This paper introduces a different Multi-Tasking Learning approach called MonoNext that utilizes a spatial grid to map objects in the scene. MonoNext employs a straightforward approach based on the ConvNext network and requires only 3D bounding box annotated data. In our experiments with the KITTI dataset, MonoNext achieved high precision and competitive performance comparable with state-of-the-art approaches. Furthermore, by adding more training data, MonoNext surpassed itself and achieved higher accuracies.
Learning to Model the World with Language

To interact with humans in the world, agents need to understand the diverse types of language that people use, relate them to the visual world, and act based on them. While current agents learn to execute simple language instructions from task rewards, we aim to build agents that leverage diverse language that conveys general knowledge, describes the state of the world, provides interactive feedback, and more. Our key idea is that language helps agents predict the future: what will be observed, how the world will behave, and which situations will be rewarded. This perspective unifies language understanding with future prediction as a powerful self-supervised learning objective. We present Dynalang, an agent that learns a multimodal world model that predicts future text and image representations and learns to act from imagined model rollouts. Unlike traditional agents that use language only to predict actions, Dynalang acquires rich language understanding by using past language also to predict future language, video, and rewards. In addition to learning from online interaction in an environment, Dynalang can be pretrained on datasets of text, video, or both without actions or rewards. From using language hints in grid worlds to navigating photorealistic scans of homes, Dynalang utilizes diverse types of language to improve task performance, including environment descriptions, game rules, and instructions.
FinVis-GPT: A Multimodal Large Language Model for Financial Chart Analysis

In this paper, we propose FinVis-GPT, a novel multimodal large language model (LLM) specifically designed for financial chart analysis. By leveraging the power of LLMs and incorporating instruction tuning and multimodal capabilities, FinVis-GPT is capable of interpreting financial charts and providing valuable analysis. To train FinVis-GPT, a financial task oriented dataset was generated for pre-training alignment and instruction tuning, comprising various types of financial charts and their corresponding descriptions. We evaluate the model performance via several case studies due to the time limit, and the promising results demonstrated that FinVis-GPT is superior in various financial chart related tasks, including generating descriptions, answering questions and predicting future market trends, surpassing existing state-of-the-art multimodal LLMs. The proposed FinVis-GPT serves as a pioneering effort in utilizing multimodal LLMs in the finance domain and our generated dataset will be release for public use in the near future to speedup related research.
Generalized Quantum Signal Processing

Quantum Signal Processing (QSP) and Quantum Singular Value Transformation (QSVT) currently stand as the most efficient techniques for implementing functions of block encoded matrices, a central task that lies at the heart of most prominent quantum algorithms. However, current QSP approaches face several challenges, such as the restrictions imposed on the family of achievable polynomials and the difficulty of calculating the required phase angles for specific transformations. In this paper, we present a Generalized Quantum Signal Processing (GQSP) approach, employing general SU(2) rotations as our signal processing operators, rather than relying solely on rotations in a single basis. Our approach lifts all practical restrictions on the family of achievable transformations, with the sole remaining condition being that $|P|\leq 1$, a restriction necessary due to the unitary nature of quantum computation. Furthermore, GQSP provides a straightforward recursive formula for determining the rotation angles needed to construct the polynomials in cases where $P$ and $Q$ are known. In cases where only $P$ is known, we provide an efficient optimization algorithm capable of identifying in under a minute of GPU time, a corresponding $Q$ for polynomials of degree on the order of $10^7$. We further illustrate GQSP simplifies QSP-based strategies for Hamiltonian simulation, offer an optimal solution to the $\epsilon$-approximate fractional query problem that requires $O(\frac{1}{\delta} + \log(\large\frac{1}{\epsilon}))$ queries to perform where $O(1/\delta)$ is a proved lower bound, and introduces novel approaches for implementing bosonic operators. Moreover, we propose a novel framework for the implementation of normal matrices, demonstrating its applicability through the development of a new convolution algorithm that runs in $O(d \log{N} + \log^2N)$ 1 and 2-qubit gates for a filter of lengths $d$.
Improved Padding in CNNs for Quantitative Susceptibility Mapping

Recently, deep learning methods have been proposed for quantitative susceptibility mapping (QSM) data processing: background field removal, field-to-source inversion, and single-step QSM reconstruction. However, the conventional padding mechanism used in convolutional neural networks (CNNs) can introduce spatial artifacts, especially in QSM background field removal and single-step QSM which requires inference from total fields with extreme large values at the edge boundaries of volume of interest. To address this issue, we propose an improved padding technique which utilizes the neighboring valid voxels to estimate the invalid voxels of feature maps at volume boundaries in the neural networks. Studies using simulated and in-vivo data show that the proposed padding greatly improves estimation accuracy and reduces artifacts in the results in the tasks of background field removal, field-to-source inversion, and single-step QSM reconstruction.