Reviewerly/exHarmony huggingface dataset:
Gigahertz Detection Rates and Dynamic Photon-Number Resolution with Superconducting Nanowire Arrays

Superconducting nanowire single-photon detectors (SNSPDs) have enabled the realization of several quantum optics technologies thanks to their high system detection efficiency (SDE), low dark counts, and fast recovery time. However, the widespread use of linear optical quantum computing, quasi-deterministic single-photon sources, and quantum repeaters requires even faster detectors that can also distinguish between different photon-number states. Here, we present an SNSPD array composed of 14 independent pixels, achieving an SDE of 90% in the telecommunications band. By reading each pixel of the array independently, we show detection of telecommunication photons at 1.5 GHz with 45% absolute SDE. We exploit the dynamic photon-number resolution of the array to demonstrate accurate state reconstruction for a wide range of light inputs, including operation with long-duration light pulses, as obtained with some cavity-based sources. We show two-photon and three-photon fidelities of 74% and 57%, respectively, which represent state-of-the-art results for fiber-coupled SNSPDs.
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles between Client Data Subspaces

Clustered federated learning (FL) has been shown to produce promising results by grouping clients into clusters. This is especially effective in scenarios where separate groups of clients have significant differences in the distributions of their local data. Existing clustered FL algorithms are essentially trying to group together clients with similar distributions so that clients in the same cluster can leverage each other's data to better perform federated learning. However, prior clustered FL algorithms attempt to learn these distribution similarities indirectly during training, which can be quite time consuming as many rounds of federated learning may be required until the formation of clusters is stabilized. In this paper, we propose a new approach to federated learning that directly aims to efficiently identify distribution similarities among clients by analyzing the principal angles between the client data subspaces. Each client applies a truncated singular value decomposition (SVD) step on its local data in a single-shot manner to derive a small set of principal vectors, which provides a signature that succinctly captures the main characteristics of the underlying distribution. This small set of principal vectors is provided to the server so that the server can directly identify distribution similarities among the clients to form clusters. This is achieved by comparing the similarities of the principal angles between the client data subspaces spanned by those principal vectors. The approach provides a simple, yet effective clustered FL framework that addresses a broad range of data heterogeneity issues beyond simpler forms of Non-IIDness like label skews. Our clustered FL approach also enables convergence guarantees for non-convex objectives.
Characterizing Entanglement Dimensionality from Randomized Measurements

The dimensionality of entanglement is probed using properties of the correlation matrix in randomized measurements, a method that works without a common reference frame and with arbitrary dimensionality.
Stereoscopic scalable quantum convolutional neural networks

As the noisy intermediate-scale quantum (NISQ) era has begun, a quantum neural network (QNN) is definitely a promising solution to many problems that classical neural networks cannot solve. In addition, a quantum convolutional neural network (QCNN) is now receiving a lot of attention because it can process high dimensional inputs comparing to QNN. However, due to the nature of quantum computing, it is difficult to scale up the QCNN to extract a sufficient number of features due to barren plateaus. This is especially challenging in classification operations with high-dimensional data input. However, due to the nature of quantum computing, it is difficult to scale up the QCNN to extract a sufficient number of features due to barren plateaus. This is especially challenging in classification operations with high dimensional data input. Motivated by this, a novel stereoscopic 3D scalable QCNN (sQCNN-3D) is proposed for point cloud data processing in classification applications. Furthermore, reverse fidelity training (RF-Train) is additionally considered on top of sQCNN-3D for diversifying features with a limited number of qubits using the fidelity of quantum computing. Our data-intensive performance evaluation verifies that the proposed algorithm achieves desired performance.
Pose-Oriented Transformer with Uncertainty-Guided Refinement for 2D-to-3D Human Pose Estimation

There has been a recent surge of interest in introducing transformers to 3D human pose estimation (HPE) due to their powerful capabilities in modeling long-term dependencies. However, existing transformer-based methods treat body joints as equally important inputs and ignore the prior knowledge of human skeleton topology in the self-attention mechanism. To tackle this issue, in this paper, we propose a Pose-Oriented Transformer (POT) with uncertainty guided refinement for 3D HPE. Specifically, we first develop novel pose-oriented self-attention mechanism and distance-related position embedding for POT to explicitly exploit the human skeleton topology. The pose-oriented self-attention mechanism explicitly models the topological interactions between body joints, whereas the distance-related position embedding encodes the distance of joints to the root joint to distinguish groups of joints with different difficulties in regression. Furthermore, we present an Uncertainty-Guided Refinement Network (UGRN) to refine pose predictions from POT, especially for the difficult joints, by considering the estimated uncertainty of each joint with uncertainty-guided sampling strategy and self-attention mechanism. Extensive experiments demonstrate that our method significantly outperforms the state-of-the-art methods with reduced model parameters on 3D HPE benchmarks such as Human3.6M and MPI-INF-3DHP.
DUET: Cross-Modal Semantic Grounding for Contrastive Zero-Shot Learning

Zero-shot learning (ZSL) aims to predict unseen classes whose samples have never appeared during training. One of the most effective and widely used semantic information for zero-shot image classification are attributes which are annotations for class-level visual characteristics. However, the current methods often fail to discriminate those subtle visual distinctions between images due to not only the shortage of fine-grained annotations, but also the attribute imbalance and co-occurrence. In this paper, we present a transformer-based end-to-end ZSL method named DUET, which integrates latent semantic knowledge from the pre-trained language models (PLMs) via a self-supervised multi-modal learning paradigm. Specifically, we (1) developed a cross-modal semantic grounding network to investigate the model's capability of disentangling semantic attributes from the images; (2) applied an attribute-level contrastive learning strategy to further enhance the model's discrimination on fine-grained visual characteristics against the attribute co-occurrence and imbalance; (3) proposed a multi-task learning policy for considering multi-model objectives. We find that our DUET can achieve state-of-the-art performance on three standard ZSL benchmarks and a knowledge graph equipped ZSL benchmark. Its components are effective and its predictions are interpretable.
Fabulation as an Approach for Design Futuring

Envisioning alternative futures and desirable worlds is a core element of design that must be cultivated, especially when a deep transition of practices, values, and power is necessary for vibrant and just future lifeworlds. In this paper, we contribute towards fabulation as an approach for design futuring that foregrounds feminist commitments and more-than-human concerns. Analyzing two fabulation case studies around biodata and bodily fluids, we offer three themes based on our process of developing these fabulations: how they engage materials, how they work to trouble temporalities, and how they cultivate imagination. We argue for the emerging potential of fabulation as an approach for open-ended, joyful design futuring, mobilizing speculative storytelling to foreground absent or neglected relations when imagining alternative lifeworlds.
High‐Dimensional Entanglement for Quantum Communication in the Frequency Domain

High-dimensional photonic entanglement is a promising candidate for error-protected quantum information processing with improved capacity. Encoding high-dimensional qudits in the carrier frequency of photons combines ease of generation, universal single-photon gates, and compatibility with fiber transmission for high-capacity quantum communication. Recent landmark experiments have impressively demonstrated quantum interference of a few frequency modes, yet the certification of massive-dimensional frequency entanglement has remained an open challenge. This study shows how to harness the large frequency-entanglement inherent in standard continuous-wave spontaneous parametric down-conversion processes. It further reports a record certification of discretized frequency entanglement, combined with a novel approach for certification that is both highly efficient and nonlocally implementable. This technique requires very few measurements and does not require assumptions on the state. The work opens the possibility for utilizing this encoding in quantum communications and in quantum information science in general.
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs).CoT explicitly encourages the LLM to generate intermediate rationales for solving a problem, by providing a series of reasoning steps in the demonstrations.Despite its success, there is still little understanding of what makes CoT prompting effective and which aspects of the demonstrated reasoning steps contribute to its performance.In this paper, we show that CoT reasoning is possible even with invalid demonstrations-prompting with invalid reasoning steps can achieve over 80-90% of the performance obtained using CoT under various metrics, while still generating coherent lines of reasoning during inference.Further experiments show that other aspects of the rationales, such as being relevant to the query and correctly ordering the reasoning steps, are much more important for effective CoT reasoning.Overall, these findings both deepen our understanding of CoT prompting, and open up new questions regarding LLMs' capability to learn to reason in context. 1
Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt

We have witnessed the rapid proliferation of multimodal data on numerous social media platforms.Conventional studies typically require massive labeled data to train models for Multimodal Aspect-Based Sentiment Analysis (MABSA).However, collecting and annotating fine-grained multimodal data for MABSA is tough.To alleviate the above issue, we perform three MABSA-related tasks with quite a small number of labeled multimodal samples.We first build diverse and comprehensive multimodal few-shot datasets according to the data distribution.To capture the specific prompt for each aspect term in a few-shot scenario, we propose a novel Generative Multimodal Prompt (GMP) 1 model for MABSA, which includes the Multimodal Encoder module and the N-Stream Decoders module.We further introduce a subtask to predict the number of aspect terms in each instance to construct the multimodal prompt.Extensive experiments on two datasets demonstrate that our approach outperforms strong baselines on two MABSA-related tasks in the few-shot setting.
In-context Examples Selection for Machine Translation

Large-scale generative models show an impressive ability to perform a wide range of Natural Language Processing (NLP) tasks using in-context learning, where a few examples are used to describe a task to the model.For Machine Translation (MT), these examples are typically randomly sampled from the development dataset with a similar distribution as the evaluation set.However, it is unclear how the choice of these in-context examples and their ordering impacts the output translation quality.In this work, we aim to understand the properties of good in-context examples for MT in both in-domain and out-of-domain settings.We show that the translation quality and the domain of the in-context examples matter and that 1-shot noisy unrelated example can have a catastrophic impact on output quality.While concatenating multiple random examples reduces the effect of noise, a single good prompt optimized to maximize translation quality on the development dataset can elicit learned information from the pre-trained language model.Adding similar examples based on an n-gram overlap with the test source significantly and consistently improves the translation quality of the outputs, outperforming a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets.
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval

In this paper, we propose SIMLM (Similarity matching with Language Model pre-training), a simple yet effective pre-training method for dense passage retrieval.It employs a simple bottleneck architecture that learns to compress the passage information into a dense vector through self-supervised pre-training.We use a replaced language modeling objective, which is inspired by ELECTRA (Clark et al., 2020), to improve the sample efficiency and reduce the mismatch of the input distribution between pre-training and fine-tuning.SIMLM only requires access to an unlabeled corpus and is more broadly applicable when there are no labeled data or queries.We conduct experiments on several large-scale passage retrieval datasets and show substantial improvements over strong baselines under various settings.Remarkably, SIMLM even outperforms multivector approaches such as ColBERTv2 (Santhanam et al., 2021) which incurs significantly more storage cost.Our code and model checkpoints are available at https:
VIRT: Improving Representation-based Text Matching via Virtual Interaction

Text matching is a fundamental research problem in natural language understanding.Interaction-based approaches treat the text pair as a single sequence and encode it through cross encoders, while representation-based models encode the text pair independently with siamese or dual encoders.Interactionbased models require dense computations and thus are impractical in real-world applications.Representation-based models have become the mainstream paradigm for efficient text matching.However, these models suffer from severe performance degradation due to the lack of interactions between the pair of texts.To remedy this, we propose a Virtual InteRacTion mechanism (VIRT) for improving representation-based text matching while maintaining its efficiency.In particular, we introduce an interactive knowledge distillation module that is only applied during training.It enables deep interaction between texts by effectively transferring knowledge from the interaction-based model.A light interaction strategy is designed to fully leverage the learned interactive knowledge.Experimental results on six text matching benchmarks demonstrate the superior performance of our method over several state-of-the-art representationbased models.We further show that VIRT can be integrated into existing methods as plugins to lift their performances.
Detection of Audio Tampering Based on Electric Network Frequency Signal

The detection of audio tampering plays a crucial role in ensuring the authenticity and integrity of multimedia files. This paper presents a novel approach to identifying tampered audio files by leveraging the unique Electric Network Frequency (ENF) signal, which is inherent to the power grid and serves as a reliable indicator of authenticity. The study begins by establishing a comprehensive Chinese ENF database containing diverse ENF signals extracted from audio files. The proposed methodology involves extracting the ENF signal, applying wavelet decomposition, and utilizing the autoregressive model to train effective classification models. Subsequently, the framework is employed to detect audio tampering and assess the influence of various environmental conditions and recording devices on the ENF signal. Experimental evaluations conducted on our Chinese ENF database demonstrate the efficacy of the proposed method, achieving impressive accuracy rates ranging from 91% to 93%. The results emphasize the significance of ENF-based approaches in enhancing audio file forensics and reaffirm the necessity of adopting reliable tamper detection techniques in multimedia authentication.
Prompt Learns Prompt: Exploring Knowledge-Aware Generative Prompt Collaboration For Video Captioning

Fine-tuning large vision-language models is a challenging task. Prompt tuning approaches have been introduced to learn fixed textual or visual prompts while freezing the pre-trained model in downstream tasks. Despite the effectiveness of prompt tuning, what do those learnable prompts learn remains unexplained. In this work, we explore whether prompts in the fine-tuning can learn knowledge-aware prompts from the pre-training, by designing two different sets of prompts in pre-training and fine-tuning phases respectively. Specifically, we present a Video-Language Prompt tuning (VL-Prompt) approach for video captioning, which first efficiently pre-train a video-language model to extract key information (e.g., actions and objects) with flexibly generated Knowledge-Aware Prompt (KAP). Then, we design a Video-Language Prompt (VLP) to transfer the knowledge from the knowledge-aware prompts and fine-tune the model to generate full captions. Experimental results show the superior performance of our approach over several state-of-the-art baselines. We further demonstrate that the video-language prompts are well learned from the knowledge-aware prompts.
TECHS: Temporal Logical Graph Networks for Explainable Extrapolation Reasoning

Extrapolation reasoning on temporal knowledge graphs (TKGs) aims to forecast future facts based on past counterparts.There are two main challenges: (1) incorporating the complex information, including structural dependencies, temporal dynamics, and hidden logical rules;(2) implementing differentiable logical rule learning and reasoning for explainability.To this end, we propose an explainable extrapolation reasoning framework TEemporal logiCal grapH networkS (TECHS), which mainly contains a temporal graph encoder and a logical decoder.The former employs a graph convolutional network with temporal encoding and heterogeneous attention to embed topological structures and temporal dynamics.The latter integrates propositional reasoning and first-order reasoning by introducing a reasoning graph that iteratively expands to find the answer.A forward message-passing mechanism is also proposed to update node representations, and their propositional and first-order attention scores.Experimental results demonstrate that it outperforms state-of-the-art baselines.
End-to-End 3D Dense Captioning with Vote2Cap-DETR

3D dense captioning aims to generate multiple captions localized with their associated object regions. Existing methods follow a sophisticated “detect-then-describe” pipeline equipped with numerous hand-crafted components. However, these hand-crafted components would yield sub-optimal performance given cluttered object spatial and class distributions among different scenes. In this paper, we propose a simple-yet-effective transformer framework Vote2Cap-DETR based on recent popular DEtection TRansformer (DETR). Compared with prior arts, our framework has several appealing advantages: 1) Without resorting to numerous hand-crafted components, our method is based on a full transformer encoder-decoder architecture with a learnable vote query driven object decoder, and a caption decoder that produces the dense captions in a set-prediction manner. 2) In contrast to the two-stage scheme, our method can perform detection and captioning in one-stage. 3) Without bells and whistles, extensive experiments on two commonly used datasets, ScanRefer and Nr3D, demonstrate that our Vote2Cap-DETR surpasses current state-of-the-arts by 11.13% and 7.11% in CIDEr@0.5IoU, respectively. Codes will be released soon.
PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation

Recently, transformer-based methods have gained significant success in sequential 2D-to-3D lifting human pose estimation. As a pioneering work, PoseFormer captures spatial relations of human joints in each video frame and human dynamics across frames with cascaded transformer layers and has achieved impressive performance. However, in real scenarios, the performance of PoseFormer and its follow-ups is limited by two factors: (a) The length of the input joint sequence; (b) The quality of 2D joint detection. Existing methods typically apply self-attention to all frames of the input sequence, causing a huge computational burden when the frame number is increased to obtain advanced estimation accuracy, and they are not robust to noise naturally brought by the limited capability of 2D joint detectors. In this paper, we propose PoseFormerV2, which exploits a compact representation of lengthy skeleton sequences in the frequency domain to efficiently scale up the receptive field and boost robustness to noisy 2D joint detection. With minimum modifications to PoseFormer, the proposed method effectively fuses features both in the time domain and frequency domain, enjoying a better speed-accuracy trade-off than its precursor. Extensive experiments on two benchmark datasets (i.e., Human3.6M and MPI-INF-3DHP) demonstrate that the proposed approach significantly outperforms the original PoseFormer and other transformer-based variants. Code is released at https://github.com/ QitaoZhao/PoseFormerV2.
Cuckoo Search Optimization-Based Bilateral Filter for Multiplicative Noise Reduction in Satellite Images

&lt;div&gt;Speckle noise degrades the visual appearance and the quality of a synthetic aperture radar (SAR) image. The reduction of speckle noise is the first step in any remote-sensing device. To improve the noisy SAR images, a variety of adaptive and nonadaptive noise reduction filters were used. In order to eliminate speckle noise present in SAR images, an adaptive cuckoo search optimization-based speckle reduction bilateral filter has been designed in this article. To test the ability to eliminate multiplicative noise, the suggested filter’s effectiveness was compared to that of several de-speckling approaches. It has been measured with different assessment metrics such as PSNR, EPI, SSIM, and ENL. When compared to conventional de-noising filters, the proposed filter shows promising results for lowering speckle noise and retaining edge properties. In addition, the PSNR value has increased as compared to the PMD method and this method has been shown to be efficient in reducing speckle noise in SAR images.&lt;/div&gt;