pwc-archive/papers-with-abstracts huggingface dataset:
Hybrid pulse propagation model and quasi-phase-matched four-wave mixing in multipass cells
Date: 2020-09-15 00:00:00

We describe a nonlinear propagation model based on a generalized Schrödinger equation in the time domain coupled to Gaussian beam evolution through ABCD matrices that account for Kerr lensing in the spatial domain. This
model is well suited to simulate propagation in mildly nonlinear systems such as multipass cells for temporal compression. It is validated against both a full(x, y, z, t) numerical model and recently reported experimental results
in multipass cells, with excellent agreement. It also allows us to identify the physical mechanism for the recently
reported parasitic appearance of spectral content in the 700–950 nm range in argon-filled multipass cells that are
used to compress pulses at 1030 nm. We think this is due to a quasi-phase-matched degenerate four-wave mixing
process. This process could be used in the future to perform wavelength conversion as is already done in fibers and
capillaries.
Deciphering Air Travel Disruptions: A Machine Learning Approach
Date: 2024-08-05 00:00:00
Tasks: regression, Time Series

This research investigates flight delay trends by examining factors such as departure time, airline, and airport. It employs regression machine learning methods to predict the contributions of various sources to delays. Time-series models, including LSTM, Hybrid LSTM, and Bi-LSTM, are compared with baseline regression models such as Multiple Regression, Decision Tree Regression, Random Forest Regression, and Neural Network. Despite considerable errors in the baseline models, the study aims to identify influential features in delay prediction, potentially informing flight planning strategies. Unlike previous work, this research focuses on regression tasks and explores the use of time-series models for predicting flight delays. It offers insights into aviation operations by independently analyzing each delay component (e.g., security, weather).
Universal Matrix Multiplication on Quantum Computer
Authors: Jiaqi Yao, Tianjian Huang, Ding Liu
Date: 2024-08-06 00:00:00

As a core underlying operation in pattern recognition and machine learning, matrix multiplication plays a crucial role in modern machine learning models and constitutes a major contributor to computational expenditure. Hence, researchers have spent decades continuously searching for more efficient matrix multiplication algorithms.This paper firstly introduces an innovative and practical approach to universal quantum matrix multiplication. We designed optimized quantum adders and multipliers based on Quantum Fourier Transform (QFT), which significantly reduced the number of gates used compared to classical adders and multipliers. Subsequently, we construct the basic universal quantum matrix multiplication and extend it to the Strassen algorithm. We conduct comparative experiments to analyze the performance of the quantum matrix multiplication and evaluate the acceleration provided by the optimized quantum adder and multiplier. Finally, we investigate the advantages of the quantum Strassen algorithm and the basic quantum matrix multiplication. Our result opens, for the first time, a reliable pathway for designing universal quantum matrix multiplication. Following this pathway, quantum computing will unlock significantly greater potential for training modern machine learning models.
OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agents
Authors: Qiang Sun, Yuanyi Luo, Sirui Li, Wenxiao Zhang, Wei Liu
Date: 2024-08-06 00:00:00
Tasks: Benchmarking, Retrieval-augmented Generation, Speech-to-Text

Multimodal conversational agents are highly desirable because they offer natural and human-like interaction. However, there is a lack of comprehensive end-to-end solutions to support collaborative development and benchmarking. While proprietary systems like GPT-4o and Gemini demonstrating impressive integration of audio, video, and text with response times of 200-250ms, challenges remain in balancing latency, accuracy, cost, and data privacy. To better understand and quantify these issues, we developed OpenOmni, an open-source, end-to-end pipeline benchmarking tool that integrates advanced technologies such as Speech-to-Text, Emotion Detection, Retrieval Augmented Generation, Large Language Models, along with the ability to integrate customized models. OpenOmni supports local and cloud deployment, ensuring data privacy and supporting latency and accuracy benchmarking. This flexible framework allows researchers to customize the pipeline, focusing on real bottlenecks and facilitating rapid proof-of-concept development. OpenOmni can significantly enhance applications like indoor assistance for visually impaired individuals, advancing human-computer interaction. Our demonstration video is available https://www.youtube.com/watch?v=zaSiT3clWqY, demo is available via https://openomni.ai4wa.com, code is available via https://github.com/AI4WA/OpenOmniFramework.
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement
Date: 2024-08-06 00:00:00
Tasks: Code Generation, Disentanglement, Instruction Following, Mathematical Reasoning

Merging Large Language Models (LLMs) aims to amalgamate multiple homologous LLMs into one with all the capabilities. Ideally, any LLMs sharing the same backbone should be mergeable, irrespective of whether they are Fine-Tuned (FT) with minor parameter changes or Pre-Trained (PT) with substantial parameter shifts. However, existing methods often manually assign the model importance, rendering them feasible only for LLMs with similar parameter alterations, such as multiple FT LLMs. The diverse parameter changed ranges between FT and PT LLMs pose challenges for current solutions in empirically determining the optimal combination. In this paper, we make a pioneering effort to broaden the applicability of merging techniques from FT to PT LLMs. We initially examine the efficacy of current methods in merging FT and PT LLMs, discovering that they struggle to deal with PT LLMs. Subsequently, we introduce an approach based on WeIght DisENtanglement (WIDEN) to effectively extend the merging scope, which first disentangles model weights into magnitude and direction components, and then performs adaptive fusion by considering their respective contributions. In the experiments, we merge Qwen1.5-Chat (an FT LLM with instruction-following skills) with Sailor (a PT LLM with multilingual abilities) across 7B and 14B model scales. Results reveal that: (1) existing solutions usually fail when merging Sailor, either losing both abilities or only retaining instruction-following skills; (2) WIDEN successfully injects the multilingual abilities of Sailor into Qwen1.5-Chat and make it proficient in Southeast Asian languages, achieving enhancements in the fundamental capabilities. In light of previous research, we also merge multiple 13B FT LLMs and observe that WIDEN achieves a balanced amalgamation of instruction following, mathematical reasoning, and code generation skills.
Visual Analysis of Multi-outcome Causal Graphs
Authors: Mengjie Fan, Jinlu Yu, Daniel Weiskopf, Nan Cao, Huai-Yu Wang, Liang Zhou
Date: 2024-07-31 00:00:00
Tasks: Causal Discovery

We introduce a visual analysis method for multiple causal graphs with different outcome variables, namely, multi-outcome causal graphs. Multi-outcome causal graphs are important in healthcare for understanding multimorbidity and comorbidity. To support the visual analysis, we collaborated with medical experts to devise two comparative visualization techniques at different stages of the analysis process. First, a progressive visualization method is proposed for comparing multiple state-of-the-art causal discovery algorithms. The method can handle mixed-type datasets comprising both continuous and categorical variables and assist in the creation of a fine-tuned causal graph of a single outcome. Second, a comparative graph layout technique and specialized visual encodings are devised for the quick comparison of multiple causal graphs. In our visual analysis approach, analysts start by building individual causal graphs for each outcome variable, and then, multi-outcome causal graphs are generated and visualized with our comparative technique for analyzing differences and commonalities of these causal graphs. Evaluation includes quantitative measurements on benchmark datasets, a case study with a medical expert, and expert user studies with real-world health research data.
LLaVA-OneVision: Easy Visual Task Transfer
Authors: Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, Chunyuan Li
Date: 2024-08-06 00:00:00
Tasks: 3D Question Answering (3D-QA), Multiple-choice, Temporal Relation Extraction, Transfer Learning, Video Question Answering, Video Understanding, Visual Question Answering, Visual Question Answering (VQA), Zero-Shot Video Question Answer

We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series. Our experimental results demonstrate that LLaVA-OneVision is the first single model that can simultaneously push the performance boundaries of open LMMs in three important computer vision scenarios: single-image, multi-image, and video scenarios. Importantly, the design of LLaVA-OneVision allows strong transfer learning across different modalities/scenarios, yielding new emerging capabilities. In particular, strong video understanding and cross-scenario capabilities are demonstrated through task transfer from images to videos.
Don't Think It Twice: Exploit Shift Invariance for Efficient Online Streaming Inference of CNNs
Date: 2024-08-06 00:00:00
Tasks: Computational Efficiency, Time Series

Deep learning time-series processing often relies on convolutional neural networks with overlapping windows. This overlap allows the network to produce an output faster than the window length. However, it introduces additional computations. This work explores the potential to optimize computational efficiency during inference by exploiting convolution's shift-invariance properties to skip the calculation of layer activations between successive overlapping windows. Although convolutions are shift-invariant, zero-padding and pooling operations, widely used in such networks, are not efficient and complicate efficient streaming inference. We introduce StreamiNNC, a strategy to deploy Convolutional Neural Networks for online streaming inference. We explore the adverse effects of zero padding and pooling on the accuracy of streaming inference, deriving theoretical error upper bounds for pooling during streaming. We address these limitations by proposing signal padding and pooling alignment and provide guidelines for designing and deploying models for StreamiNNC. We validate our method in simulated data and on three real-world biomedical signal processing applications. StreamiNNC achieves a low deviation between streaming output and normal inference for all three networks (2.03 - 3.55% NRMSE). This work demonstrates that it is possible to linearly speed up the inference of streaming CNNs processing overlapping windows, negating the additional computation typically incurred by overlapping windows.
CAFE-Net: Cross-Attention and Feature Exploration Network for polyp segmentation
Date: 2024-03-15 00:00:00

Colorectal polyp segmentation can help physicians screen colonoscopy images, which is essential for preventing colorectal cancer. The segmentation of polyps encounters multiple challenges, like small size, uneven brightness, blurred edges, and the potential confusion of folds with objects. Although existing methods have shown promising performance in addressing these challenges, there are three main shortcomings: (1) during the segmentation process, small polyp objects are lost, (2) the decoder stage faces challenges in restoring fine-grained details of the features, and (3) the limited capacity to aggregate multi-scale features. We propose a cross-attention and feature exploration network (CAFE-Net) for polyp segmentation to address these challenges. The work offers the following contributions: (1) a feature supplement and exploration module (FSEM) supplements in missing details and explores latent features, (2) a cross-attention decoder module (CADM) effectively preserves features from lower layers and restores fine-grained information, and (3) a multi-scale feature aggregation (MFA) module maximizes the utilization of previously learned features. We conducted extensive experiments and compared CAFE-Net with nine state-of-the-art (SOTA) methods. The CAFE-Net demonstrates the best segmentation accuracy across multiple polyp datasets as well as has an obvious advantage in segmenting small polyp objects.
MDM: Advancing Multi-Domain Distribution Matching for Automatic Modulation Recognition Dataset Synthesis
Date: 2024-08-05 00:00:00
Tasks: Automatic Modulation Recognition, Dataset Distillation

Recently, deep learning technology has been successfully introduced into Automatic Modulation Recognition (AMR) tasks. However, the success of deep learning is all attributed to the training on large-scale datasets. Such a large amount of data brings huge pressure on storage, transmission and model training. In order to solve the problem of large amount of data, some researchers put forward the method of data distillation, which aims to compress large training data into smaller synthetic datasets to maintain its performance. While numerous data distillation techniques have been developed within the realm of image processing, the unique characteristics of signals set them apart. Signals exhibit distinct features across various domains, necessitating specialized approaches for their analysis and processing. To this end, a novel dataset distillation method--Multi-domain Distribution Matching (MDM) is proposed. MDM employs the Discrete Fourier Transform (DFT) to translate timedomain signals into the frequency domain, and then uses a model to compute distribution matching losses between the synthetic and real datasets, considering both the time and frequency domains. Ultimately, these two losses are integrated to update the synthetic dataset. We conduct extensive experiments on three AMR datasets. Experimental results show that, compared with baseline methods, our method achieves better performance under the same compression ratio. Furthermore, we conduct crossarchitecture generalization experiments on several models, and the experimental results show that our synthetic datasets can generalize well on other unseen models.
BodySLAM: A Generalized Monocular Visual SLAM Framework for Surgical Applications
Authors: G. Manni, C. Lauretti, F. Prata, R. Papalia, L. Zollo, P. Soda
Date: 2024-08-06 00:00:00
Tasks: 3D Reconstruction, Depth Estimation, Monocular Depth Estimation, Pose Estimation, Simultaneous Localization and Mapping

Endoscopic surgery relies on two-dimensional views, posing challenges for surgeons in depth perception and instrument manipulation. While Monocular Visual Simultaneous Localization and Mapping (MVSLAM) has emerged as a promising solution, its implementation in endoscopic procedures faces significant challenges due to hardware limitations, such as the use of a monocular camera and the absence of odometry sensors. This study presents BodySLAM, a robust deep learning-based MVSLAM approach that addresses these challenges through three key components: CycleVO, a novel unsupervised monocular pose estimation module; the integration of the state-of-the-art Zoe architecture for monocular depth estimation; and a 3D reconstruction module creating a coherent surgical map. The approach is rigorously evaluated using three publicly available datasets (Hamlyn, EndoSLAM, and SCARED) spanning laparoscopy, gastroscopy, and colonoscopy scenarios, and benchmarked against four state-of-the-art methods. Results demonstrate that CycleVO exhibited competitive performance with the lowest inference time among pose estimation methods, while maintaining robust generalization capabilities, whereas Zoe significantly outperformed existing algorithms for depth estimation in endoscopy. BodySLAM's strong performance across diverse endoscopic scenarios demonstrates its potential as a viable MVSLAM solution for endoscopic applications.
Training LLMs to Recognize Hedges in Spontaneous Narratives
Date: 2024-08-06 00:00:00
Tasks: Attribute

Hedges allow speakers to mark utterances as provisional, whether to signal non-prototypicality or "fuzziness", to indicate a lack of commitment to an utterance, to attribute responsibility for a statement to someone else, to invite input from a partner, or to soften critical feedback in the service of face-management needs. Here we focus on hedges in an experimentally parameterized corpus of 63 Roadrunner cartoon narratives spontaneously produced from memory by 21 speakers for co-present addressees, transcribed to text (Galati and Brennan, 2010). We created a gold standard of hedges annotated by human coders (the Roadrunner-Hedge corpus) and compared three LLM-based approaches for hedge detection: fine-tuning BERT, and zero and few-shot prompting with GPT-4o and LLaMA-3. The best-performing approach was a fine-tuned BERT model, followed by few-shot GPT-4o. After an error analysis on the top performing approaches, we used an LLM-in-the-Loop approach to improve the gold standard coding, as well as to highlight cases in which hedges are ambiguous in linguistically interesting ways that will guide future research. This is the first step in our research program to train LLMs to interpret and generate collateral signals appropriately and meaningfully in conversation.
Fusing Forces: Deep-Human-Guided Refinement of Segmentation Masks
Date: 2024-08-06 00:00:00

Etruscan mirrors constitute a significant category in Etruscan art, characterized by elaborate figurative illustrations featured on their backside. A laborious and costly aspect of their analysis and documentation is the task of manually tracing these illustrations. In previous work, a methodology has been proposed to automate this process, involving photometric-stereo scanning in combination with deep neural networks. While achieving quantitative performance akin to an expert annotator, some results still lack qualitative precision and, thus, require annotators for inspection and potential correction, maintaining resource intensity. In response, we propose a deep neural network trained to interactively refine existing annotations based on human guidance. Our human-in-the-loop approach streamlines annotation, achieving equal quality with up to 75% less manual input required. Moreover, during the refinement process, the relative improvement of our methodology over pure manual labeling reaches peak values of up to 26%, attaining drastically better quality quicker. By being tailored to the complex task of segmenting intricate lines, specifically distinguishing it from previous methods, our approach offers drastic improvements in efficacy, transferable to a broad spectrum of applications beyond Etruscan mirrors.
Understanding How Blind Users Handle Object Recognition Errors: Strategies and Challenges
Date: 2024-08-06 00:00:00
Tasks: Object, Object Recognition

Object recognition technologies hold the potential to support blind and low-vision people in navigating the world around them. However, the gap between benchmark performances and practical usability remains a significant challenge. This paper presents a study aimed at understanding blind users' interaction with object recognition systems for identifying and avoiding errors. Leveraging a pre-existing object recognition system, URCam, fine-tuned for our experiment, we conducted a user study involving 12 blind and low-vision participants. Through in-depth interviews and hands-on error identification tasks, we gained insights into users' experiences, challenges, and strategies for identifying errors in camera-based assistive technologies and object recognition systems. During interviews, many participants preferred independent error review, while expressing apprehension toward misrecognitions. In the error identification task, participants varied viewpoints, backgrounds, and object sizes in their images to avoid and overcome errors. Even after repeating the task, participants identified only half of the errors, and the proportion of errors identified did not significantly differ from their first attempts. Based on these insights, we offer implications for designing accessible interfaces tailored to the needs of blind and low-vision users in identifying object recognition errors.
LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theory
Authors: Jillian Ross, Yoon Kim, Andrew W. Lo
Date: 2024-08-05 00:00:00
Tasks: Decision Making

Humans are not homo economicus (i.e., rational economic beings). As humans, we exhibit systematic behavioral biases such as loss aversion, anchoring, framing, etc., which lead us to make suboptimal economic decisions. Insofar as such biases may be embedded in text data on which large language models (LLMs) are trained, to what extent are LLMs prone to the same behavioral biases? Understanding these biases in LLMs is crucial for deploying LLMs to support human decision-making. We propose utility theory-a paradigm at the core of modern economic theory-as an approach to evaluate the economic biases of LLMs. Utility theory enables the quantification and comparison of economic behavior against benchmarks such as perfect rationality or human behavior. To demonstrate our approach, we quantify and compare the economic behavior of a variety of open- and closed-source LLMs. We find that the economic behavior of current LLMs is neither entirely human-like nor entirely economicus-like. We also find that most current LLMs struggle to maintain consistent economic behavior across settings. Finally, we illustrate how our approach can measure the effect of interventions such as prompting on economic biases.
Examining Gender and Power on Wikipedia Through Face and Politeness
Authors: Adil Soubki, Shyne Choi, Owen Rambow
Date: 2024-08-05 00:00:00

We propose a framework for analyzing discourse by combining two interdependent concepts from sociolinguistic theory: face acts and politeness. While politeness has robust existing tools and data, face acts are less resourced. We introduce a new corpus created by annotating Wikipedia talk pages with face acts and we use this to train a face act tagger. We then employ our framework to study how face and politeness interact with gender and power in discussions between Wikipedia editors. Among other findings, we observe that female Wikipedians are not only more polite, which is consistent with prior studies, but that this difference corresponds with significantly more language directed at humbling aspects of their own face. Interestingly, the distinction nearly vanishes once limiting to editors with administrative power.