pwc-archive/papers-with-abstracts huggingface dataset:
A Python Framework for Neutrosophic Sets and Mappings
Authors: Giorgio Nordo, Saeid Jafari, Arif Mehmood, Bhimraj Basumatary
Date: 2024-03-24 00:00:00

In this paper we present an open source framework developed in Python and consisting of three distinct classes designed to manipulate in a simple and intuitive way both symbolic representations of neutrosophic sets over universes of various types as well as mappings between them. The capabilities offered by this framework extend and generalize previous attempts to provide software solutions to the manipulation of neutrosophic sets such as those proposed by Salama et al., Saranya et al., El-Ghareeb, Topal et al. and Sleem. The code is described in detail and many examples and use cases are also provided.
Soil respiration signals in response to sustainable soil management practices enhance soil organic carbon stocks
Authors: Mario Guevara
Date: 2024-03-28 00:00:00
Tasks: Management

Development of a spatial-temporal and data-driven model of soil respiration at the global scale based on soil temperature, yearly soil moisture, and soil organic carbon (C) estimates. Prediction of soil respiration on an annual basis (1991-2018) with relatively high accuracy (NSE 0.69, CCC 0.82). Lower soil respiration trends, higher soil respiration magnitudes, and higher soil organic C stocks across areas experiencing the presence of sustainable soil management practices.
Denoising Variational Graph of Graphs Auto-Encoder for Predicting Structured Entity Interactions
Authors: PDF Han Chen, Hanchen Wang, Hongmei Chen, Ying Zhang, Wenjie Zhang, Xuemin Lin
Date: 2023-07-24 00:00:00
Proceeding: IEEE Transactions on Knowledge and Data Engineering 2023 7
Tasks: Denoising

The interactions between structured entities play important roles in a wide range of applications such as chemistry, material science, biology, and medical science. Recently, graph-based methods have been exploited to effectively predict the interactions among structured entities. However, these methods usually only focus on structural information of the entities and are incapable of fully utilizing the interaction information between the entities. In this paper, we propose a Denoising Variational Graph of Graphs Auto-encoder (DVGGA) which follows the graph of graphs framework to capture both structural information in structured entities and interaction information among structured entities. With denoising criterion, DVGGA is able to capture the information from the useful structures of the local graph and address the overfitting issue caused by redundant substructures. Extensive experiments conducted on real-world datasets show that DVGGA outperforms the state-of-the-art structured entity interaction prediction methods.
RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalization
Authors: Zhecheng Yuan, Sizhe Yang, Pu Hua, Can Chang, Kaizhe Hu, Huazhe Xu
Date: 2023-07-15 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Out-of-Distribution Generalization, reinforcement-learning, Reinforcement Learning

Visual Reinforcement Learning (Visual RL), coupled with high-dimensional observations, has consistently confronted the long-standing challenge of out-of-distribution generalization. Despite the focus on algorithms aimed at resolving visual generalization problems, we argue that the devil is in the existing benchmarks as they are restricted to isolated tasks and generalization categories, undermining a comprehensive evaluation of agents' visual generalization capabilities. To bridge this gap, we introduce RL-ViGen: a novel Reinforcement Learning Benchmark for Visual Generalization, which contains diverse tasks and a wide spectrum of generalization types, thereby facilitating the derivation of more reliable conclusions. Furthermore, RL-ViGen incorporates the latest generalization visual RL algorithms into a unified framework, under which the experiment results indicate that no single existing algorithm has prevailed universally across tasks. Our aspiration is that RL-ViGen will serve as a catalyst in this area, and lay a foundation for the future creation of universal visual generalization RL agents suitable for real-world scenarios. Access to our code and implemented algorithms is provided at https://gemcollector.github.io/RL-ViGen/.
Implicit Multi-Spectral Transformer: An Lightweight and Effective Visible to Infrared Image Translation Model
Authors: Yijia Chen, Pinghua Chen, Xiangxin Zhou, Yingtie Lei, Ziyang Zhou, Mingxian Li
Date: 2024-04-10 00:00:00
Tasks: Benchmarking, Image-to-Image Translation

In the field of computer vision, visible light images often exhibit low contrast in low-light conditions, presenting a significant challenge. While infrared imagery provides a potential solution, its utilization entails high costs and practical limitations. Recent advancements in deep learning, particularly the deployment of Generative Adversarial Networks (GANs), have facilitated the transformation of visible light images to infrared images. However, these methods often experience unstable training phases and may produce suboptimal outputs. To address these issues, we propose a novel end-to-end Transformer-based model that efficiently converts visible light images into high-fidelity infrared images. Initially, the Texture Mapping Module and Color Perception Adapter collaborate to extract texture and color features from the visible light image. The Dynamic Fusion Aggregation Module subsequently integrates these features. Finally, the transformation into an infrared image is refined through the synergistic action of the Color Perception Adapter and the Enhanced Perception Attention mechanism. Comprehensive benchmarking experiments confirm that our model outperforms existing methods, producing infrared images of markedly superior quality, both qualitatively and quantitatively. Furthermore, the proposed model enables more effective downstream applications for infrared images than other methods.
Emotion-cause pair extraction method based on multi-granularity information and multi-module interaction
Authors: Mingrui Fu, Weijiang Li
Date: 2024-04-10 00:00:00
Tasks: Emotion-Cause Pair Extraction, Position

The purpose of emotion-cause pair extraction is to extract the pair of emotion clauses and cause clauses. On the one hand, the existing methods do not take fully into account the relationship between the emotion extraction of two auxiliary tasks. On the other hand, the existing two-stage model has the problem of error propagation. In addition, existing models do not adequately address the emotion and cause-induced locational imbalance of samples. To solve these problems, an end-to-end multitasking model (MM-ECPE) based on shared interaction between GRU, knowledge graph and transformer modules is proposed. Furthermore, based on MM-ECPE, in order to use the encoder layer to better solve the problem of imbalanced distribution of clause distances between clauses and emotion clauses, we propose a novel encoding based on BERT, sentiment lexicon, and position-aware interaction module layer of emotion motif pair retrieval model (MM-ECPE(BERT)). The model first fully models the interaction between different tasks through the multi-level sharing module, and mines the shared information between emotion-cause pair extraction and the emotion extraction and cause extraction. Second, to solve the imbalanced distribution of emotion clauses and cause clauses problem, suitable labels are screened out according to the knowledge graph path length and task-specific features are constructed so that the model can focus on extracting pairs with corresponding emotion-cause relationships. Experimental results on the ECPE benchmark dataset show that the proposed model achieves good performance, especially on position-imbalanced samples.
GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models
Authors: Zewei Zhang, Huan Liu, Jun Chen, Xiangyu Xu
Date: 2024-04-10 00:00:00
Tasks: Benchmarking, Denoising

In this paper, we introduce GoodDrag, a novel approach to improve the stability and image quality of drag editing. Unlike existing methods that struggle with accumulated perturbations and often result in distortions, GoodDrag introduces an AlDD framework that alternates between drag and denoising operations within the diffusion process, effectively improving the fidelity of the result. We also propose an information-preserving motion supervision operation that maintains the original features of the starting point for precise manipulation and artifact reduction. In addition, we contribute to the benchmarking of drag editing by introducing a new dataset, Drag100, and developing dedicated quality assessment metrics, Dragging Accuracy Index and Gemini Score, utilizing Large Multimodal Models. Extensive experiments demonstrate that the proposed GoodDrag compares favorably against the state-of-the-art approaches both qualitatively and quantitatively. The project page is https://gooddrag.github.io.
BRAVE: Broadening the visual encoding of vision-language models
Authors: Oğuzhan Fatih Kar, Alessio Tonioni, Petra Poklukar, Achin Kulshrestha, Amir Zamir, Federico Tombari
Date: 2024-04-10 00:00:00
Tasks: Hallucination, Language Modelling, Visual Question Answering (VQA)

Vision-language models (VLMs) are typically composed of a vision encoder, e.g. CLIP, and a language model (LM) that interprets the encoded features to solve downstream tasks. Despite remarkable progress, VLMs are subject to several shortcomings due to the limited capabilities of vision encoders, e.g. "blindness" to certain image features, visual hallucination, etc. To address these issues, we study broadening the visual encoding capabilities of VLMs. We first comprehensively benchmark several vision encoders with different inductive biases for solving VLM tasks. We observe that there is no single encoding configuration that consistently achieves top performance across different tasks, and encoders with different biases can perform surprisingly similarly. Motivated by this, we introduce a method, named BRAVE, that consolidates features from multiple frozen encoders into a more versatile representation that can be directly fed as the input to a frozen LM. BRAVE achieves state-of-the-art performance on a broad range of captioning and VQA benchmarks and significantly reduces the aforementioned issues of VLMs, while requiring a smaller number of trainable parameters than existing methods and having a more compressed representation. Our results highlight the potential of incorporating different visual biases for a more broad and contextualized visual understanding of VLMs.
UMBRAE: Unified Multimodal Brain Decoding
Authors: Weihao Xia, Raoul de Charette, Cengiz Öztireli, Jing-Hao Xue
Date: 2024-04-10 00:00:00
Tasks: Brain Decoding, Language Modeling, Language Modelling, Large Language Model, Multimodal Large Language Model

We address prevailing challenges of the brain-powered research, departing from the observation that the literature hardly recover accurate spatial information and require subject-specific models. To address these challenges, we propose UMBRAE, a unified multimodal decoding of brain signals. First, to extract instance-level conceptual and spatial details from neural signals, we introduce an efficient universal brain encoder for multimodal-brain alignment and recover object descriptions at multiple levels of granularity from subsequent multimodal large language model (MLLM). Second, we introduce a cross-subject training strategy mapping subject-specific features to a common feature space. This allows a model to be trained on multiple subjects without extra resources, even yielding superior results compared to subject-specific models. Further, we demonstrate this supports weakly-supervised adaptation to new subjects, with only a fraction of the total training data. Experiments demonstrate that UMBRAE not only achieves superior results in the newly introduced tasks but also outperforms methods in well established tasks. To assess our method, we construct and share with the community a comprehensive brain understanding benchmark BrainHub. Our code and benchmark are available at https://weihaox.github.io/UMBRAE.
RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion
Authors: Jaidev Shriram, Alex Trevithick, Lingjie Liu, Ravi Ramamoorthi
Date: 2024-04-10 00:00:00
Tasks: 3D Inpainting, Scene Generation

We introduce RealmDreamer, a technique for generating forward-facing 3D scenes from text descriptions. Our method optimizes a 3D Gaussian Splatting representation to match complex text prompts using pretrained diffusion models. Our key insight is to leverage 2D inpainting diffusion models conditioned on an initial scene estimate to provide low variance supervision for unknown regions during 3D distillation. In conjunction, we imbue high-fidelity geometry with geometric distillation from a depth diffusion model, conditioned on samples from the inpainting model. We find that the initialization of the optimization is crucial, and provide a principled methodology for doing so. Notably, our technique doesn't require video or multi-view data and can synthesize various high-quality 3D scenes in different styles with complex layouts. Further, the generality of our method allows 3D synthesis from a single image. As measured by a comprehensive user study, our method outperforms all existing approaches, preferred by 88-95%. Project Page: https://realmdreamer.github.io/
A Foundation Model for Zero-shot Logical Query Reasoning
Authors: Mikhail Galkin, Jincheng Zhou, Bruno Ribeiro, Jian Tang, Zhaocheng Zhu
Date: 2024-04-10 00:00:00
Tasks: Complex Query Answering, Knowledge Graph Completion, Knowledge Graphs

Complex logical query answering (CLQA) in knowledge graphs (KGs) goes beyond simple KG completion and aims at answering compositional queries comprised of multiple projections and logical operations. Existing CLQA methods that learn parameters bound to certain entity or relation vocabularies can only be applied to the graph they are trained on which requires substantial training time before being deployed on a new graph. Here we present UltraQuery, the first foundation model for inductive reasoning that can zero-shot answer logical queries on any KG. The core idea of UltraQuery is to derive both projections and logical operations as vocabulary-independent functions which generalize to new entities and relations in any KG. With the projection operation initialized from a pre-trained inductive KG reasoning model, UltraQuery can solve CLQA on any KG after finetuning on a single dataset. Experimenting on 23 datasets, UltraQuery in the zero-shot inference mode shows competitive or better query answering performance than best available baselines and sets a new state of the art on 15 of them.
VN-EGNN: E(3)-Equivariant Graph Neural Networks with Virtual Nodes Enhance Protein Binding Site Identification
Authors: Florian Sestak, Lisa Schneckenreiter, Johannes Brandstetter, Sepp Hochreiter, Andreas Mayr, Günter Klambauer
Date: 2024-04-10 00:00:00
Tasks: Trajectory Prediction

Being able to identify regions within or around proteins, to which ligands can potentially bind, is an essential step to develop new drugs. Binding site identification methods can now profit from the availability of large amounts of 3D structures in protein structure databases or from AlphaFold predictions. Current binding site identification methods heavily rely on graph neural networks (GNNs), usually designed to output E(3)-equivariant predictions. Such methods turned out to be very beneficial for physics-related tasks like binding energy or motion trajectory prediction. However, the performance of GNNs at binding site identification is still limited potentially due to the lack of dedicated nodes that model hidden geometric entities, such as binding pockets. In this work, we extend E(n)-Equivariant Graph Neural Networks (EGNNs) by adding virtual nodes and applying an extended message passing scheme. The virtual nodes in these graphs are dedicated quantities to learn representations of binding sites, which leads to improved predictive performance. In our experiments, we show that our proposed method VN-EGNN sets a new state-of-the-art at locating binding site centers on COACH420, HOLO4K and PDBbind2020.
Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery
Authors: Zohre Karimi, Shing-Hei Ho, Bao Thach, Alan Kuntz, Daniel S. Brown
Date: 2024-04-10 00:00:00
Tasks: Decision Making, Imitation Learning, Reinforcement Learning (RL), Sequential Decision Making

Automating robotic surgery via learning from demonstration (LfD) techniques is extremely challenging. This is because surgical tasks often involve sequential decision-making processes with complex interactions of physical objects and have low tolerance for mistakes. Prior works assume that all demonstrations are fully observable and optimal, which might not be practical in the real world. This paper introduces a sample-efficient method that learns a robust reward function from a limited amount of ranked suboptimal demonstrations consisting of partial-view point cloud observations. The method then learns a policy by optimizing the learned reward function using reinforcement learning (RL). We show that using a learned reward function to obtain a policy is more robust than pure imitation learning. We apply our approach on a physical surgical electrocautery task and demonstrate that our method can perform well even when the provided demonstrations are suboptimal and the observations are high-dimensional point clouds. Code and videos available here: https://sites.google.com/view/lfdinelectrocautery
Worst-Case Convergence Time of ML Algorithms via Extreme Value Theory
Authors: Saeid Tizpaz-Niari, Sriram Sankaranarayanan
Date: 2024-04-10 00:00:00

This paper leverages the statistics of extreme values to predict the worst-case convergence times of machine learning algorithms. Timing is a critical non-functional property of ML systems, and providing the worst-case converge times is essential to guarantee the availability of ML and its services. However, timing properties such as worst-case convergence times (WCCT) are difficult to verify since (1) they are not encoded in the syntax or semantics of underlying programming languages of AI, (2) their evaluations depend on both algorithmic implementations and underlying systems, and (3) their measurements involve uncertainty and noise. Therefore, prevalent formal methods and statistical models fail to provide rich information on the amounts and likelihood of WCCT. Our key observation is that the timing information we seek represents the extreme tail of execution times. Therefore, extreme value theory (EVT), a statistical discipline that focuses on understanding and predicting the distribution of extreme values in the tail of outcomes, provides an ideal framework to model and analyze WCCT in the training and inference phases of ML paradigm. Building upon the mathematical tools from EVT, we propose a practical framework to predict the worst-case timing properties of ML. Over a set of linear ML training algorithms, we show that EVT achieves a better accuracy for predicting WCCTs than relevant statistical methods such as the Bayesian factor. On the set of larger machine learning training algorithms and deep neural network inference, we show the feasibility and usefulness of EVT models to accurately predict WCCTs, their expected return periods, and their likelihood.
Using Neural Networks to Model Hysteretic Kinematics in Tendon-Actuated Continuum Robots
Authors: YuAn Wang, Max McCandless, Abdulhamit Donder, Giovanni Pittiglio, Behnam Moradkhani, Yash Chitalia, Pierre E. Dupont
Date: 2024-04-10 00:00:00

The ability to accurately model mechanical hysteretic behavior in tendon-actuated continuum robots using deep learning approaches is a growing area of interest. In this paper, we investigate the hysteretic response of two types of tendon-actuated continuum robots and, ultimately, compare three types of neural network modeling approaches with both forward and inverse kinematic mappings: feedforward neural network (FNN), FNN with a history input buffer, and long short-term memory (LSTM) network. We seek to determine which model best captures temporal dependent behavior. We find that, depending on the robot's design, choosing different kinematic inputs can alter whether hysteresis is exhibited by the system. Furthermore, we present the results of the model fittings, revealing that, in contrast to the standard FNN, both FNN with a history input buffer and the LSTM model exhibit the capacity to model historical dependence with comparable performance in capturing rate-dependent hysteresis.
PIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory System
Authors: Steve Rhyner, Haocong Luo, Juan Gómez-Luna, Mohammad Sadrosadati, Jiawei Jiang, Ataberk Olgun, Harshita Gupta, Ce Zhang, Onur Mutlu
Date: 2024-04-10 00:00:00
Tasks: CPU, Distributed Optimization, GPU

Modern Machine Learning (ML) training on large-scale datasets is a very time-consuming workload. It relies on the optimization algorithm Stochastic Gradient Descent (SGD) due to its effectiveness, simplicity, and generalization performance. Processor-centric architectures (e.g., CPUs, GPUs) commonly used for modern ML training workloads based on SGD are bottlenecked by data movement between the processor and memory units due to the poor data locality in accessing large datasets. As a result, processor-centric architectures suffer from low performance and high energy consumption while executing ML training workloads. Processing-In-Memory (PIM) is a promising solution to alleviate the data movement bottleneck by placing the computation mechanisms inside or near memory. Our goal is to understand the capabilities of popular distributed SGD algorithms on real-world PIM systems to accelerate data-intensive ML training workloads. To this end, we 1) implement several representative centralized parallel SGD algorithms on the real-world UPMEM PIM system, 2) rigorously evaluate these algorithms for ML training on large-scale datasets in terms of performance, accuracy, and scalability, 3) compare to conventional CPU and GPU baselines, and 4) discuss implications for future PIM hardware and highlight the need for a shift to an algorithm-hardware codesign. Our results demonstrate three major findings: 1) The UPMEM PIM system can be a viable alternative to state-of-the-art CPUs and GPUs for many memory-bound ML training workloads, especially when operations and datatypes are natively supported by PIM hardware, 2) it is important to carefully choose the optimization algorithms that best fit PIM, and 3) the UPMEM PIM system does not scale approximately linearly with the number of nodes for many data-intensive ML training workloads. We open source all our code to facilitate future research.
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Authors: Tsendsuren Munkhdalai, Manaal Faruqui, Siddharth Gopal
Date: 2024-04-10 00:00:00
Tasks: Book summarization, Language Modeling, Language Modelling, Retrieval

This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation. A key component in our proposed approach is a new attention technique dubbed Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. We demonstrate the effectiveness of our approach on long-context language modeling benchmarks, 1M sequence length passkey context block retrieval and 500K length book summarization tasks with 1B and 8B LLMs. Our approach introduces minimal bounded memory parameters and enables fast streaming inference for LLMs.