pwc-archive/papers-with-abstracts huggingface dataset:
Interactions with Generative Information Retrieval Systems
Authors: Mohammad Aliannejadi, Jacek Gwizdka, Hamed Zamani
Date: 2024-07-16 00:00:00
Tasks: Information Retrieval, Retrieval

At its core, information access and seeking is an interactive process. In existing search engines, interactions are limited to a few pre-defined actions, such as "requery", "click on a document", "scrolling up/down", "going to the next result page", "leaving the search engine", etc. A major benefit of moving towards generative IR systems is enabling users with a richer expression of information need and feedback and free-form interactions in natural language and beyond. In other words, the actions users take are no longer limited by the clickable links and buttons available on the search engine result page and users can express themselves freely through natural language. This can go even beyond natural language, through images, videos, gestures, and sensors using multi-modal generative IR systems. This chapter briefly discusses the role of interaction in generative IR systems. We will first discuss different ways users can express their information needs by interacting with generative IR systems. We then explain how users can provide explicit or implicit feedback to generative IR systems and how they can consume such feedback. Next, we will cover how users interactively can refine retrieval results. We will expand upon mixed-initiative interactions and discuss clarification and preference elicitation in more detail. We then discuss proactive generative IR systems, including context-aware recommendation, following up past conversations, contributing to multi-party conversations, and feedback requests. Providing explanation is another interaction type that we briefly discuss in this chapter. We will also briefly describe multi-modal interactions in generative information retrieval. Finally, we describe emerging frameworks and solutions for user interfaces with generative AI systems.
A PLMs based protein retrieval framework
Authors: Yuxuan Wu, Xiao Yi, Yang Tan, Huiqun Yu, Guisheng Fan, Gaowei Zheng
Date: 2024-07-16 00:00:00
Tasks: Retrieval

Protein retrieval, which targets the deconstruction of the relationship between sequences, structures and functions, empowers the advancing of biology. Basic Local Alignment Search Tool (BLAST), a sequence-similarity-based algorithm, has proved the efficiency of this field. Despite the existing tools for protein retrieval, they prioritize sequence similarity and probably overlook proteins that are dissimilar but share homology or functionality. In order to tackle this problem, we propose a novel protein retrieval framework that mitigates the bias towards sequence similarity. Our framework initiatively harnesses protein language models (PLMs) to embed protein sequences within a high-dimensional feature space, thereby enhancing the representation capacity for subsequent analysis. Subsequently, an accelerated indexed vector database is constructed to facilitate expedited access and retrieval of dense vectors. Extensive experiments demonstrate that our framework can equally retrieve both similar and dissimilar proteins. Moreover, this approach enables the identification of proteins that conventional methods fail to uncover. This framework will effectively assist in protein mining and empower the development of biology.
Bootstrapped Pre-training with Dynamic Identifier Prediction for Generative Retrieval
Authors: Yubao Tang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
Date: 2024-07-16 00:00:00
Tasks: Memorization, Retrieval

Generative retrieval uses differentiable search indexes to directly generate relevant document identifiers in response to a query. Recent studies have highlighted the potential of a strong generative retrieval model, trained with carefully crafted pre-training tasks, to enhance downstream retrieval tasks via fine-tuning. However, the full power of pre-training for generative retrieval remains underexploited due to its reliance on pre-defined static document identifiers, which may not align with evolving model parameters. In this work, we introduce BootRet, a bootstrapped pre-training method for generative retrieval that dynamically adjusts document identifiers during pre-training to accommodate the continuing memorization of the corpus. BootRet involves three key training phases: (i) initial identifier generation, (ii) pre-training via corpus indexing and relevance prediction tasks, and (iii) bootstrapping for identifier updates. To facilitate the pre-training phase, we further introduce noisy documents and pseudo-queries, generated by large language models, to resemble semantic connections in both indexing and retrieval tasks. Experimental results demonstrate that BootRet significantly outperforms existing pre-training generative retrieval baselines and performs well even in zero-shot settings.
A kernel for time series based on global alignments
Authors: Marco Cuturi, Jean-Philippe Vert, Oystein Birkenes, Tomoko Matsui
Date: 2006-10-06 00:00:00
Tasks: Dynamic Time Warping, speech-recognition, Speech Recognition, Time Series

We propose in this paper a new family of kernels to handle times series, notably speech data, within the framework of kernel methods which includes popular algorithms such as the Support Vector Machine. These kernels elaborate on the well known Dynamic Time Warping (DTW) family of distances by considering the same set of elementary operations, namely substitutions and repetitions of tokens, to map a sequence onto another. Associating to each of these operations a given score, DTW algorithms use dynamic programming techniques to compute an optimal sequence of operations with high overall score. In this paper we consider instead the score spanned by all possible alignments, take a smoothed version of their maximum and derive a kernel out of this formulation. We prove that this kernel is positive definite under favorable conditions and show how it can be tuned effectively for practical applications as we report encouraging results on a speech recognition task.
Nowcasting R&D Expenditures: A Machine Learning Approach
Authors: Atin Aboutorabi, Gaétan de Rassenfosse
Date: 2024-07-16 00:00:00

Macroeconomic data are crucial for monitoring countries' performance and driving policy. However, traditional data acquisition processes are slow, subject to delays, and performed at a low frequency. We address this 'ragged-edge' problem with a two-step framework. The first step is a supervised learning model predicting observed low-frequency figures. We propose a neural-network-based nowcasting model that exploits mixed-frequency, high-dimensional data. The second step uses the elasticities derived from the previous step to interpolate unobserved high-frequency figures. We apply our method to nowcast countries' yearly research and development (R&D) expenditure series. These series are collected through infrequent surveys, making them ideal candidates for this task. We exploit a range of predictors, chiefly Internet search volume data, and document the relevance of these data in improving out-of-sample predictions. Furthermore, we leverage the high frequency of our data to derive monthly estimates of R&D expenditures, which are currently unobserved. We compare our results with those obtained from the classical regression-based and the sparse temporal disaggregation methods. Finally, we validate our results by reporting a strong correlation with monthly R&D employment data.
M-Net: A Lightweight Network Based on Multilayer Perceptron for Massive MIMO CSI Feedback
Authors: Yaxin Yu, Yinglei Teng, Binghui Wang, An Liu, Vincent Lau
Date: 2023-12-20 00:00:00
Proceeding: IEEE Globecom Workshops (GC Wkshps) 2023 12
Tasks: Decoder

In FDD massive multiple-input multiple-output (MIMO) systems, user equipments (UEs) are required to send back the downlink channel state information (CSI) to base stations (BSs). However, the high-dimentional CSI souring proportional to antennas leads to unacceptable feedback overhead. Recently, deep learning (DL) has shown its unique powerful ability to compress and reconstruct CSI. In this paper, a novel Multilayer Perceptron (MLP) based feedback network, named M-Net, is proposed, where the specific spatial correlation of channels is emphasized by tailored feature processing modules. To meet the practical requirements of lightweight networks and ensure low inference latency, M-Net adopts parameter sharing to reduce network complexity and eliminates complex operations. Additionally, to leverage the computational resources at the BSs, we design a scalable decoder to further improve the performance of the network. Simulation results show that the proposed M-Net achieves almost the state-of-the-art (SOTA) performance with the lowest computational complexity, while exhibiting a remarkable 49.63% reduction in the inference delay compared to TransNet. Finally, we demonstrate that the M-Net variants achieve the SOTA performance by only deep enning the M-Net decoder. The open source codes are available at https://github.com/Password-YYX/M-Net.
Towards Neuro-Symbolic Video Understanding
Authors: Minkyu Choi, Harsh Goel, Mohammad Omama, Yunhao Yang, Sahil Shah, Sandeep Chinchali
Date: 2024-03-16 00:00:00
Tasks: Video Understanding

The unprecedented surge in video data production in recent years necessitates efficient tools to extract meaningful frames from videos for downstream tasks. Long-term temporal reasoning is a key desideratum for frame retrieval systems. While state-of-the-art foundation models, like VideoLLaMA and ViCLIP, are proficient in short-term semantic understanding, they surprisingly fail at long-term reasoning across frames. A key reason for this failure is that they intertwine per-frame perception and temporal reasoning into a single deep network. Hence, decoupling but co-designing semantic understanding and temporal reasoning is essential for efficient scene identification. We propose a system that leverages vision-language models for semantic understanding of individual frames but effectively reasons about the long-term evolution of events using state machines and temporal logic (TL) formulae that inherently capture memory. Our TL-based reasoning improves the F1 score of complex event identification by 9-15% compared to benchmarks that use GPT4 for reasoning on state-of-the-art self-driving datasets such as Waymo and NuScenes.
AI-Driven Physics-Informed Bio-Silicon Intelligence System: Integrating Hybrid Systems, Biocomputing, Neural Networks, and Machine Learning, for Advanced Neurotechnology
Authors: Vincent Jorgsson, Raghav Kumar, Mustaf Ahmed, Maxx Yung, Aryaman Pattnayak, Sri Pradhyumna Sridhar, Vaishnav Varma, Arun Ram Ponnambalam, Georg Weidlich, Dimitris Pinotsis
Date: 2024-07-12 00:00:00
Tasks: Hierarchical Reinforcement Learning

We present the Bio-Silicon Intelligence System (BSIS), an innovative hybrid platform that integrates biological neural networks with silicon-based computing. The BSIS, a Physics-Informed Hybrid Hierarchical Reinforcement Learning State Machine, employs carbon nanotube-coated electrodes to interface rat brains with computational systems, enabling high-fidelity neural interfacing and bidirectional communication through self-organizing systems in both biological and silicon forms. Our system leverages both analogue and digital AI theory, incorporating concepts from computational theory, chaos theory, dynamical systems theory, physics, and quantum mechanics. Additionally, the BSIS replicates the neuronal dynamics typical of intelligent brain tissue, employing nonlinear operations underlying learning and information storage. Neural signals are read through the FreeEEG32 board and BrainFlow software, then features are extracted and mapped to game actions by tracking feature changes in continuous data. Metadata is encoded into both analogue and digital brain stimulation signals at the microvolt level using our proprietary software and hardware. The system employs a dual signaling approach for training the rat brain, incorporating a reward solution and sound as well as human-inaudible distress sounds. This paper details the design, theory, functionality, and technical specifications of the BSIS, highlighting its interdisciplinary approach and advanced technological integration.
Efficacy of Various Large Language Models in Generating Smart Contracts
Authors: Siddhartha Chatterjee, Bina Ramamurthy
Date: 2024-06-28 00:00:00
Tasks: Code Generation

This study analyzes the application of code-generating Large Language Models in the creation of immutable Solidity smart contracts on the Ethereum Blockchain. Other works have previously analyzed Artificial Intelligence code generation abilities. This paper aims to expand this to a larger scope to include programs where security and efficiency are of utmost priority such as smart contracts. The hypothesis leading into the study was that LLMs in general would have difficulty in rigorously implementing security details in the code, which was shown through our results, but surprisingly generally succeeded in many common types of contracts. We also discovered a novel way of generating smart contracts through new prompting strategies.
VCP-CLIP: A visual context prompting model for zero-shot anomaly segmentation
Authors: Zhen Qu, Xian Tao, Mukesh Prasad, Fei Shen, Zhengtao Zhang, Xinyi Gong, Guiguang Ding
Date: 2024-07-17 00:00:00
Tasks: Anomaly Detection, Anomaly Segmentation, zero-shot anomaly detection, Zero Shot Segmentation

Recently, large-scale vision-language models such as CLIP have demonstrated immense potential in zero-shot anomaly segmentation (ZSAS) task, utilizing a unified model to directly detect anomalies on any unseen product with painstakingly crafted text prompts. However, existing methods often assume that the product category to be inspected is known, thus setting product-specific text prompts, which is difficult to achieve in the data privacy scenarios. Moreover, even the same type of product exhibits significant differences due to specific components and variations in the production process, posing significant challenges to the design of text prompts. In this end, we propose a visual context prompting model (VCP-CLIP) for ZSAS task based on CLIP. The insight behind VCP-CLIP is to employ visual context prompting to activate CLIP's anomalous semantic perception ability. In specific, we first design a Pre-VCP module to embed global visual information into the text prompt, thus eliminating the necessity for product-specific prompts. Then, we propose a novel Post-VCP module, that adjusts the text embeddings utilizing the fine-grained features of the images. In extensive experiments conducted on 10 real-world industrial anomaly segmentation datasets, VCP-CLIP achieved state-of-the-art performance in ZSAS task. The code is available at https://github.com/xiaozhen228/VCP-CLIP.
SignSpeak: Open-Source Time Series Classification for ASL Translation
Authors: Aditya Makkar, Divya Makkar, Aarav Patel, Liam Hebert
Date: 2024-06-27 00:00:00
Tasks: Time Series, Time Series Classification, Translation

The lack of fluency in sign language remains a barrier to seamless communication for hearing and speech-impaired communities. In this work, we propose a low-cost, real-time ASL-to-speech translation glove and an exhaustive training dataset of sign language patterns. We then benchmarked this dataset with supervised learning models, such as LSTMs, GRUs and Transformers, where our best model achieved 92% accuracy. The SignSpeak dataset has 7200 samples encompassing 36 classes (A-Z, 1-10) and aims to capture realistic signing patterns by using five low-cost flex sensors to measure finger positions at each time step at 36 Hz. Our open-source dataset, models and glove designs, provide an accurate and efficient ASL translator while maintaining cost-effectiveness, establishing a framework for future work to build on.
Serialized Point Mamba: A Serialized Point Cloud Mamba Segmentation Model
Authors: Tao Wang, Wei Wen, Jingzhi Zhai, Kang Xu, Haoming Luo
Date: 2024-07-17 00:00:00
Tasks: 3D Instance Segmentation, 3D Reconstruction, 3D Semantic Segmentation, Computational Efficiency, Instance Segmentation, LIDAR Semantic Segmentation, Mamba, Point Cloud Segmentation, Segmentation, Semantic Segmentation

Point cloud segmentation is crucial for robotic visual perception and environmental understanding, enabling applications such as robotic navigation and 3D reconstruction. However, handling the sparse and unordered nature of point cloud data presents challenges for efficient and accurate segmentation. Inspired by the Mamba model's success in natural language processing, we propose the Serialized Point Cloud Mamba Segmentation Model (Serialized Point Mamba), which leverages a state-space model to dynamically compress sequences, reduce memory usage, and enhance computational efficiency. Serialized Point Mamba integrates local-global modeling capabilities with linear complexity, achieving state-of-the-art performance on both indoor and outdoor datasets. This approach includes novel techniques such as staged point cloud sequence learning, grid pooling, and Conditional Positional Encoding, facilitating effective segmentation across diverse point cloud tasks. Our method achieved 76.8 mIoU on Scannet and 70.3 mIoU on S3DIS. In Scannetv2 instance segmentation, it recorded 40.0 mAP. It also had the lowest latency and reasonable memory use, making it the SOTA among point semantic segmentation models based on mamba.
CDFL: Efficient Federated Human Activity Recognition using Contrastive Learning and Deep Clustering
Authors: Ensieh Khazaei, Alireza Esmaeilzehi, Bilal Taha, Dimitrios Hatzinakos
Date: 2024-07-17 00:00:00
Tasks: Activity Recognition, Clustering, Contrastive Learning, Deep Clustering, Federated Learning, Human Activity Recognition

In the realm of ubiquitous computing, Human Activity Recognition (HAR) is vital for the automation and intelligent identification of human actions through data from diverse sensors. However, traditional machine learning approaches by aggregating data on a central server and centralized processing are memory-intensive and raise privacy concerns. Federated Learning (FL) has emerged as a solution by training a global model collaboratively across multiple devices by exchanging their local model parameters instead of local data. However, in realistic settings, sensor data on devices is non-independently and identically distributed (Non-IID). This means that data activity recorded by most devices is sparse, and sensor data distribution for each client may be inconsistent. As a result, typical FL frameworks in heterogeneous environments suffer from slow convergence and poor performance due to deviation of the global model's objective from the global objective. Most FL methods applied to HAR are either designed for overly ideal scenarios without considering the Non-IID problem or present privacy and scalability concerns. This work addresses these challenges, proposing CDFL, an efficient federated learning framework for image-based HAR. CDFL efficiently selects a representative set of privacy-preserved images using contrastive learning and deep clustering, reduces communication overhead by selecting effective clients for global model updates, and improves global model quality by training on privacy-preserved data. Our comprehensive experiments carried out on three public datasets, namely Stanford40, PPMI, and VOC2012, demonstrate the superiority of CDFL in terms of performance, convergence rate, and bandwidth usage compared to state-of-the-art approaches.
Deep Learning-based Sentiment Analysis of Olympics Tweets
Authors: Indranil Bandyopadhyay, Rahul Karmakar
Date: 2024-07-17 00:00:00
Tasks: Deep Learning, Sentiment Analysis, Sentiment Classification

Sentiment analysis (SA), is an approach of natural language processing (NLP) for determining a text's emotional tone by analyzing subjective information such as views, feelings, and attitudes toward specific topics, products, services, events, or experiences. This study attempts to develop an advanced deep learning (DL) model for SA to understand global audience emotions through tweets in the context of the Olympic Games. The findings represent global attitudes around the Olympics and contribute to advancing the SA models. We have used NLP for tweet pre-processing and sophisticated DL models for arguing with SA, this research enhances the reliability and accuracy of sentiment classification. The study focuses on data selection, preprocessing, visualization, feature extraction, and model building, featuring a baseline Na\"ive Bayes (NB) model and three advanced DL models: Convolutional Neural Network (CNN), Bidirectional Long Short-Term Memory (BiLSTM), and Bidirectional Encoder Representations from Transformers (BERT). The results of the experiments show that the BERT model can efficiently classify sentiments related to the Olympics, achieving the highest accuracy of 99.23%.
OMG-Net: A Deep Learning Framework Deploying Segment Anything to Detect Pan-Cancer Mitotic Figures from Haematoxylin and Eosin-Stained Slides
Authors: Zhuoyan Shen, Mikael Simard, Douglas Brand, Vanghelita Andrei, Ali Al-Khader, Fatine Oumlil, Katherine Trevers, Thomas Butters, Simon Haefliger, Eleanna Kara, Fernanda Amary, Roberto Tirabosco, Paul Cool, Gary Royle, Maria A. Hawkins, Adrienne M. Flanagan, Charles-Antoine Collins Fekete
Date: 2024-07-17 00:00:00
Tasks: Breast Cancer Detection, whole slide images

Mitotic activity is an important feature for grading several cancer types. Counting mitotic figures (MFs) is a time-consuming, laborious task prone to inter-observer variation. Inaccurate recognition of MFs can lead to incorrect grading and hence potential suboptimal treatment. In this study, we propose an artificial intelligence (AI)-aided approach to detect MFs in digitised haematoxylin and eosin-stained whole slide images (WSIs). Advances in this area are hampered by the limited number and types of cancer datasets of MFs. Here we establish the largest pan-cancer dataset of mitotic figures by combining an in-house dataset of soft tissue tumours (STMF) with five open-source mitotic datasets comprising multiple human cancers and canine specimens (ICPR, TUPAC, CCMCT, CMC and MIDOG++). This new dataset identifies 74,620 MFs and 105,538 mitotic-like figures. We then employed a two-stage framework (the Optimised Mitoses Generator Network (OMG-Net) to classify MFs. The framework first deploys the Segment Anything Model (SAM) to automate the contouring of MFs and surrounding objects. An adapted ResNet18 is subsequently trained to classify MFs. OMG-Net reaches an F1-score of 0.84 on pan-cancer MF detection (breast carcinoma, neuroendocrine tumour and melanoma), largely outperforming the previous state-of-the-art MIDOG++ benchmark model on its hold-out testing set (e.g. +16% F1-score on breast cancer detection, p<0.001) thereby providing superior accuracy in detecting MFs on various types of tumours obtained with different scanners.