pwc-archive/papers-with-abstracts huggingface dataset:
A Vision-based Solution for Track Misalignment Detection
Authors: Koteswar Rao Jerripothula, Sharik Ali Ansari, Rahul Nijhawan
Date: 2021-12-20 00:00:00
Proceeding: Conference on Graphics, Patterns and Images (SIBGRAPI) 2021 12
Tasks: image-classification, Image Classification, Railway Track Image Classification, Transfer Learning

Derailment is one of the most frequent ways railway accidents happen. Track defects such as buckling and hogging that cause misalignment of tracks can easily lead to derailments. While railway tracks get laterally misaligned due to buckling, vertical misalignments can result from hogging. Such misalignments are visibly recognizable, and we can even automate the recognition using data-driven models. This paper discusses how we build such data-driven models. There are no public datasets available to build such models; therefore, we introduce TMD (Track Misalignment Detection) dataset. It consists of misaligned and normal track images. The problem we try to solve here is essentially a binary image classification problem, which we solve by exploring the feature extraction approach to transfer learning (TL). In this approach, we employ a pre-trained network to extract rich features, which are then supplied with annotations to a learning algorithm for building a candidate TL model. Several pre-trained networks and learning algorithms exist, resulting in many TL models; therefore, it becomes essential to identify effective ones. We propose an evaluation criterion to decide which are effective ones even before we test them. Our experiments demonstrate that the TL models selected based on our proposed evaluation criterion really perform better than other candidate TL models during testing.
RIS-aided OTFS Modulation in High-Doppler Channels
Authors: Gandhodi Harshavardhan, Vighnesh S Bhat, and A. Chockalingam
Date: 2022-12-20 00:00:00
Proceeding: IEEE 33rd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) 2022 12

Two research topics, namely, reconfigurable intelligent
surfaces (RIS) and orthogonal time frequency space (OTFS)
modulation, have drawn significant interest in recent wireless
research. While RIS technology can aid communication by favorably
controlling the electromagnetic wave propagation towards
the receiver, OTFS modulation can offer robust performance
in high-Doppler channels. Both RIS and OTFS are envisioned
to be promising technologies for 6G and beyond. This paper
investigates a RIS-aided OTFS system. A novel and useful
contribution in this paper is the detailed derivation of the endto-
end delay-Doppler (DD) domain input-output relation of a
RIS-aided OTFS system, which can serve as a basic system
model that can aid the investigation of efficient transceiver techniques/
algorithms, performance evaluation, and implementation.
We consider maximization of the Frobenius norm of the effective
end-to-end DD channel matrix of the RIS-aided OTFS system to
choose the phase vector at the RIS. Our simulation results show
that RIS-aided OTFS performs significantly better compared to
RIS-aided OFDM.
Index Terms—Reconfigurable intelligent surface, OTFS modulation,
delay-Doppler domain, RIS-aided OTFS, input-output
relation, RIS-aided OFDM.
Compton backscattered and primary X-rays from solar flares: angle dependent Green's function correction for photospheric albedo
Authors: Eduard P. Kontar, Alec L. MacKinnon, Richard A. Schwartz, John C. Brown
Date: 2005-10-06 00:00:00

The observed hard X-ray (HXR) flux spectrum $I(\epsilon)$ from solar flares is a combination of primary bremsstrahlung photons $I_P(\epsilon)$ with a spectrally modified component from photospheric Compton backscatter of downward primary emission. The latter can be significant, distorting or hiding the true features of the primary spectrum which are key diagnostics for acceleration and propagation of high energy electrons and of their energy budget. For the first time in solar physics, we use a Green's function approach to the backscatter spectral deconvolution problem, constructing a Green's matrix including photoelectric absorption. This approach allows spectrum-independent extraction of the primary spectrum for several HXR flares observed by the {\it Ramaty High Energy Solar Spectroscopic Imager} (RHESSI). We show that the observed and primary spectra differ very substantially for flares with hard spectra close to the disk centre. We show in particular that the energy dependent photon spectral index $\gamma (\epsilon)=-d \log I/d \log \epsilon$ is very different for $I_P(\epsilon)$ and for $I(\epsilon)$ and that inferred mean source electron spectra ${\bar F}(E)$ differ greatly. Even for a forward fitting of a parametric ${\bar F}(E)$ to the data, a clear low-energy cutoff required to fit $I(\epsilon)$ essentially disappears when the fit is to $I_P(\epsilon)$ - i.e. when albedo correction is included. The self-consistent correction for backscattered photons is thus shown to be crucial in determining the energy spectra of flare accelerated electrons, and hence their total number and energy.
COLA: A Benchmark for Compositional Text-to-image Retrieval
Authors: Arijit Ray, Filip Radenovic, Abhimanyu Dubey, Bryan A. Plummer, Ranjay Krishna, Kate Saenko
Date: 2023-05-05 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Attribute, CoLA, Image Retrieval, Retrieval

Compositional reasoning is a hallmark of human visual intelligence. Yet, despite the size of large vision-language models, they struggle to represent simple compositions by combining objects with their attributes. To measure this lack of compositional capability, we design Cola, a text-to-image retrieval benchmark to Compose Objects Localized with Attributes. To solve Cola, a model must retrieve images with the correct configuration of attributes and objects and avoid choosing a distractor image with the same objects and attributes but in the wrong configuration. Cola contains about 1.2k composed queries of 168 objects and 197 attributes on around 30K images. Our human evaluation finds that Cola is 83.33% accurate, similar to contemporary compositionality benchmarks. Using Cola as a testbed, we explore empirical modeling designs to adapt pre-trained vision-language models to reason compositionally. We explore 6 adaptation strategies on 2 seminal vision-language models, using compositionality-centric test benchmarks - Cola and CREPE. We find the optimal adaptation strategy is to train a multi-modal attention layer that jointly attends over the frozen pre-trained image and language features. Surprisingly, training multimodal layers on CLIP performs better than tuning a larger FLAVA model with already pre-trained multimodal layers. Furthermore, our adaptation strategy improves CLIP and FLAVA to comparable levels, suggesting that training multimodal layers using contrastive attribute-object data is key, as opposed to using them pre-trained. Lastly, we show that Cola is harder than a closely related contemporary benchmark, CREPE, since simpler fine-tuning strategies without multimodal layers suffice on CREPE but not on Cola. However, we still see a significant gap between our best adaptation and human accuracy, suggesting considerable room for further research.
Improving training time and GPU utilization in geo-distributed language model training
Authors: Palak, Rohan Gandhi, Karan Tandon, Debopam Bhattacherjee, Venkata N. Padmanabhan
Date: 2024-11-16 00:00:00
Tasks: GPU, Language Modeling, Language Modelling

The widespread adoption of language models (LMs) across multiple industries has caused huge surge in demand for GPUs. Training LMs requires tens of thousands of GPUs and housing them in the same datacenter (DCs) is becoming challenging. We focus on training such models across multiple DCs connected via Wide-Area-Network (WAN). We build ATLAS that speeds up such training time using novel temporal bandwidth sharing and many other design choices. While ATLAS improves the training time, it does not eliminate the bubbles (idle GPU cycles). We built BUBBLETEA that runs prefill-as-a-service (part of LM inference) during the bubbles that improves the GPU utilization substantially without any impact of training. Together, ATLAS and BUBBLETEA improve training time by up to 17X and achieve GPU utilization of up to 94%.
mR$^2$AG: Multimodal Retrieval-Reflection-Augmented Generation for Knowledge-Based VQA
Authors: Tao Zhang, Ziqi Zhang, Zongyang Ma, Yuxin Chen, Zhongang Qi, Chunfeng Yuan, Bing Li, Junfu Pu, Yuxuan Zhao, Zehua Xie, Jin Ma, Ying Shan, Weiming Hu
Date: 2024-11-22 00:00:00
Tasks: RAG, Retrieval, Retrieval-augmented Generation, Visual Question Answering (VQA)

Advanced Multimodal Large Language Models (MLLMs) struggle with recent Knowledge-based VQA tasks, such as INFOSEEK and Encyclopedic-VQA, due to their limited and frozen knowledge scope, often leading to ambiguous and inaccurate responses. Thus, multimodal Retrieval-Augmented Generation (mRAG) is naturally introduced to provide MLLMs with comprehensive and up-to-date knowledge, effectively expanding the knowledge scope. However, current mRAG methods have inherent drawbacks, including: 1) Performing retrieval even when external knowledge is not needed. 2) Lacking of identification of evidence that supports the query. 3) Increasing model complexity due to additional information filtering modules or rules. To address these shortcomings, we propose a novel generalized framework called \textbf{m}ultimodal \textbf{R}etrieval-\textbf{R}eflection-\textbf{A}ugmented \textbf{G}eneration (mR$^2$AG), which achieves adaptive retrieval and useful information localization to enable answers through two easy-to-implement reflection operations, preventing high model complexity. In mR$^2$AG, Retrieval-Reflection is designed to distinguish different user queries and avoids redundant retrieval calls, and Relevance-Reflection is introduced to guide the MLLM in locating beneficial evidence of the retrieved content and generating answers accordingly. In addition, mR$^2$AG can be integrated into any well-trained MLLM with efficient fine-tuning on the proposed mR$^2$AG Instruction-Tuning dataset (mR$^2$AG-IT). mR$^2$AG significantly outperforms state-of-the-art MLLMs (e.g., GPT-4v/o) and RAG-based MLLMs on INFOSEEK and Encyclopedic-VQA, while maintaining the exceptional capabilities of base MLLMs across a wide range of Visual-dependent tasks.
A mixed gas concentration regression prediction method based on RESHA-ALW
Authors: Jilong Wu a, Wenlong Zhao a, Fan Wu a, Jia Yan a, B, Peter Feng c, Hao Cui a, Shukai Duan a, D, Xiaoyan Peng
Date: 2024-11-01 00:00:00
Proceeding: Sensors and Actuators: B. Chemical 2024 11
Tasks: regression

Previous research in gas regression predominantly concentrated on individual gas. However, in real world, gas 
typically coexists in mixed forms and the detection technology normally involves metal oxide sensors. This work 
introduces an efffcient model, denoted as RESHA-ALW, for mixed gas concentration prediction. A hybrid 
attention (HA) model is added into Residual Neural Network (RESNET) to reweight the features by generating a 
channel attention weight matrix and a locational attention weight matrix. Furthermore, an adaptive loss 
weighting (ALW) method is also applied to achieve the dynamic weighting of the different gases during the 
training process. The performance of the model is veriffed through the ffve-fold cross-validation experiments on 
the carbon monoxide (CO)-ethylene gas mixture dataset, with mean results of 0.25, 1.78, and 0.998 for ethylene 
(RMSE, SAMPE, and R2
), as well as 7.97, 2.27, and 0.998 for CO, respectively, outperforming all the compared 
models. Then, the ALW method is applied to the comparison models, resulting in signiffcant increase in prediction
performance of all the models, which conffrms the efffciency of the ALW method. Moreover, RESHA-ALW 
has demonstrated better performance in predicting the concentration of the mixture within a single task, as 
compared to RESHA’s performance when undertaking two separate tasks. This further validates the signiffcant 
application potential of our proposed model. Finally, we also conduct ablation experiments and compare the 
impact of choosing different activation functions on the model.
Dynamical system prediction from sparse observations using deep neural networks with Voronoi tessellation and physics constraint
Authors: HanYang Wang, Hao Zhou, Sibo Cheng
Date: 2024-08-31 00:00:00
Tasks: Computational Efficiency, Prediction, Time Series Prediction

Despite the success of various methods in addressing the issue of spatial reconstruction of dynamical systems with sparse observations, spatio-temporal prediction for sparse fields remains a challenge. Existing Kriging-based frameworks for spatio-temporal sparse field prediction fail to meet the accuracy and inference time required for nonlinear dynamic prediction problems. In this paper, we introduce the Dynamical System Prediction from Sparse Observations using Voronoi Tessellation (DSOVT) framework, an innovative methodology based on Voronoi tessellation which combines convolutional encoder-decoder (CED) and long short-term memory (LSTM) and utilizing Convolutional Long Short-Term Memory (ConvLSTM). By integrating Voronoi tessellations with spatio-temporal deep learning models, DSOVT is adept at predicting dynamical systems with unstructured, sparse, and time-varying observations. CED-LSTM maps Voronoi tessellations into a low-dimensional representation for time series prediction, while ConvLSTM directly uses these tessellations in an end-to-end predictive model. Furthermore, we incorporate physics constraints during the training process for dynamical systems with explicit formulas. Compared to purely data-driven models, our physics-based approach enables the model to learn physical laws within explicitly formulated dynamics, thereby enhancing the robustness and accuracy of rolling forecasts. Numerical experiments on real sea surface data and shallow water systems clearly demonstrate our framework's accuracy and computational efficiency with sparse and time-varying observations.
FastGrasp: Efficient Grasp Synthesis with Diffusion
Authors: Xiaofei Wu, Tao Liu, Caoji Li, Yuexin Ma, Yujiao Shi, Xuming He
Date: 2024-11-22 00:00:00
Tasks: Diversity

Effectively modeling the interaction between human hands and objects is challenging due to the complex physical constraints and the requirement for high generation efficiency in applications. Prior approaches often employ computationally intensive two-stage approaches, which first generate an intermediate representation, such as contact maps, followed by an iterative optimization procedure that updates hand meshes to capture the hand-object relation. However, due to the high computation complexity during the optimization stage, such strategies often suffer from low efficiency in inference. To address this limitation, this work introduces a novel diffusion-model-based approach that generates the grasping pose in a one-stage manner. This allows us to significantly improve generation speed and the diversity of generated hand poses. In particular, we develop a Latent Diffusion Model with an Adaptation Module for object-conditioned hand pose generation and a contact-aware loss to enforce the physical constraints between hands and objects. Extensive experiments demonstrate that our method achieves faster inference, higher diversity, and superior pose quality than state-of-the-art approaches. Code is available at \href{https://github.com/wuxiaofei01/FastGrasp}{https://github.com/wuxiaofei01/FastGrasp.}
The BS-meter: A ChatGPT-Trained Instrument to Detect Sloppy Language-Games
Authors: Alessandro Trevisan, Harry Giddens, Sarah Dillon, Alan F. Blackwell
Date: 2024-11-22 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model

What can we learn about language from studying how it is used by ChatGPT and other large language model (LLM)-based chatbots? In this paper, we analyse the distinctive character of language generated by ChatGPT, in relation to questions raised by natural language processing pioneer, and student of Wittgenstein, Margaret Masterman. Following frequent complaints that LLM-based chatbots produce "slop," or even "bullshit," in the sense of Frankfurt's popular monograph On Bullshit, we conduct an empirical study to contrast the language of 1,000 scientific publications with typical text generated by ChatGPT. We then explore whether the same language features can be detected in two well-known contexts of social dysfunction: George Orwell's critique of political speech, and David Graeber's characterisation of bullshit jobs. Using simple hypothesis-testing methods, we demonstrate that a statistical model of sloppy bullshit can reliably relate the Frankfurtian artificial bullshit of ChatGPT to the political and workplace functions of bullshit as observed in natural human language.
Health AI Developer Foundations
Authors: Atilla P. Kiraly, Sebastien Baur, Kenneth Philbrick, Fereshteh Mahvar, Liron Yatziv, Tiffany Chen, Bram Sterling, Nick George, Fayaz Jamil, Jing Tang, Kai Bailey, Faruk Ahmed, Akshay Goel, Abbi Ward, Lin Yang, Andrew Sellergren, Yossi Matias, Avinatan Hassidim, Shravya Shetty, Daniel Golden, Shekoofeh Azizi, David F. Steiner, Yun Liu, Tim Thelin, Rory Pilgrim, Can Kirmizibayrak
Date: 2024-11-22 00:00:00
Tasks: Fairness

Robust medical Machine Learning (ML) models have the potential to revolutionize healthcare by accelerating clinical research, improving workflows and outcomes, and producing novel insights or capabilities. Developing such ML models from scratch is cost prohibitive and requires substantial compute, data, and time (e.g., expert labeling). To address these challenges, we introduce Health AI Developer Foundations (HAI-DEF), a suite of pre-trained, domain-specific foundation models, tools, and recipes to accelerate building ML for health applications. The models cover various modalities and domains, including radiology (X-rays and computed tomography), histopathology, dermatological imaging, and audio. These models provide domain specific embeddings that facilitate AI development with less labeled data, shorter training times, and reduced computational costs compared to traditional approaches. In addition, we utilize a common interface and style across these models, and prioritize usability to enable developers to integrate HAI-DEF efficiently. We present model evaluations across various tasks and conclude with a discussion of their application and evaluation, covering the importance of ensuring efficacy, fairness, and equity. Finally, while HAI-DEF and specifically the foundation models lower the barrier to entry for ML in healthcare, we emphasize the importance of validation with problem- and population-specific data for each desired usage setting. This technical report will be updated over time as more modalities and features are added.
ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generation
Authors: Xiaoman Zhang, Hong-Yu Zhou, Xiaoli Yang, Oishi Banerjee, Julián N. Acosta, Josh Miller, Ouwen Huang, Pranav Rajpurkar
Date: 2024-11-22 00:00:00

AI-driven models have demonstrated significant potential in automating radiology report generation for chest X-rays. However, there is no standardized benchmark for objectively evaluating their performance. To address this, we present ReXrank, https://rexrank.ai, a public leaderboard and challenge for assessing AI-powered radiology report generation. Our framework incorporates ReXGradient, the largest test dataset consisting of 10,000 studies, and three public datasets (MIMIC-CXR, IU-Xray, CheXpert Plus) for report generation assessment. ReXrank employs 8 evaluation metrics and separately assesses models capable of generating only findings sections and those providing both findings and impressions sections. By providing this standardized evaluation framework, ReXrank enables meaningful comparisons of model performance and offers crucial insights into their robustness across diverse clinical settings. Beyond its current focus on chest X-rays, ReXrank's framework sets the stage for comprehensive evaluation of automated reporting across the full spectrum of medical imaging.
VideoRepair: Improving Text-to-Video Generation via Misalignment Evaluation and Localized Refinement
Authors: Daeun Lee, Jaehong Yoon, Jaemin Cho, Mohit Bansal
Date: 2024-11-22 00:00:00
Tasks: Text-to-Video Generation, Video Alignment, Video Generation

Recent text-to-video (T2V) diffusion models have demonstrated impressive generation capabilities across various domains. However, these models often generate videos that have misalignments with text prompts, especially when the prompts describe complex scenes with multiple objects and attributes. To address this, we introduce VideoRepair, a novel model-agnostic, training-free video refinement framework that automatically identifies fine-grained text-video misalignments and generates explicit spatial and textual feedback, enabling a T2V diffusion model to perform targeted, localized refinements. VideoRepair consists of four stages: In (1) video evaluation, we detect misalignments by generating fine-grained evaluation questions and answering those questions with MLLM. In (2) refinement planning, we identify accurately generated objects and then create localized prompts to refine other areas in the video. Next, in (3) region decomposition, we segment the correctly generated area using a combined grounding module. We regenerate the video by adjusting the misaligned regions while preserving the correct regions in (4) localized refinement. On two popular video generation benchmarks (EvalCrafter and T2V-CompBench), VideoRepair substantially outperforms recent baselines across various text-video alignment metrics. We provide a comprehensive analysis of VideoRepair components and qualitative examples.
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
Authors: Hjalmar Wijk, Tao Lin, Joel Becker, Sami Jawhar, Neev Parikh, Thomas Broadley, Lawrence Chan, Michael Chen, Josh Clymer, Jai Dhyani, Elena Ericheva, Katharyn Garcia, Brian Goodrich, Nikola Jurkovic, Megan Kinniment, Aron Lajko, Seraphina Nix, Lucas Sato, William Saunders, Maksym Taran, Ben West, Elizabeth Barnes
Date: 2024-11-22 00:00:00
Tasks: AI Agent, Language Modeling, Language Modelling

Frontier AI safety policies highlight automation of AI research and development (R&D) by AI agents as an important capability to anticipate. However, there exist few evaluations for AI R&D capabilities, and none that are highly realistic and have a direct comparison to human performance. We introduce RE-Bench (Research Engineering Benchmark, v1), which consists of 7 challenging, open-ended ML research engineering environments and data from 71 8-hour attempts by 61 distinct human experts. We confirm that our experts make progress in the environments given 8 hours, with 82% of expert attempts achieving a non-zero score and 24% matching or exceeding our strong reference solutions. We compare humans to several public frontier models through best-of-k with varying time budgets and agent designs, and find that the best AI agents achieve a score 4x higher than human experts when both are given a total time budget of 2 hours per environment. However, humans currently display better returns to increasing time budgets, narrowly exceeding the top AI agent scores given an 8-hour budget, and achieving 2x the score of the top AI agent when both are given 32 total hours (across different attempts). Qualitatively, we find that modern AI agents possess significant expertise in many ML topics -- e.g. an agent wrote a faster custom Triton kernel than any of our human experts' -- and can generate and test solutions over ten times faster than humans, at much lower cost. We open-source the evaluation environments, human expert data, analysis code and agent trajectories to facilitate future research.