OpenRaiser/Intern-Atlas huggingface dataset:
UniMedVL: Unifying Medical Multimodal Understanding And Generation Through Observation-Knowledge-Analysis
Venue: arXiv cs.CV 2025
Year: 2025

Medical diagnostic applications require models that can process multimodal medical inputs (images, patient histories, lab results) and generate diverse outputs including both textual reports and visual content (annotations, segmentation masks, and images). Despite this need, existing medical AI systems disrupt this unified process: medical image understanding models interpret images but cannot generate visual outputs, while medical image generation models synthesize images but cannot provide textual explanations. This leads to gaps in data representation, feature integration, and task-level multimodal capabilities. To this end, we propose a multi-level framework that draws inspiration from diagnostic workflows through the Observation-Knowledge-Analysis (OKA) paradigm. Specifically, at the observation level, we construct UniMed-5M, a dataset comprising over 5.6M samples that reformat diverse unimodal data into multimodal pairs for foundational observation. At the knowledge level, we propose Progressive Curriculum Learning that systematically introduces medical multimodal knowledge. At the analysis level, we introduce UniMedVL, the first medical unified multimodal model for the simultaneous analysis of image understanding and generation tasks within a single architecture. UniMedVL achieves superior performance on five medical image understanding benchmarks, while matching specialized models in generation quality across eight medical imaging modalities. Crucially, our unified architecture enables bidirectional knowledge sharing: generation tasks enhance visual understanding features, demonstrating that integrating traditionally separate capabilities within a single medical framework unlocks improvements across diverse medical vision-language tasks. Code is available at https://github.com/uni-medical/UniMedVL.
Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
Venue: arXiv cs.AI 2025
Year: 2025

Reinforcement Learning from Human Feedback (RLHF) has become central to aligning large language models with human values, typically by first learning a reward model from preference data which is then used to update the model with reinforcement learning. Recent alternatives such as Direct Preference Optimization (DPO) simplify this pipeline by directly optimizing on preferences. However, both approaches often assume uniform annotator preferences and rely on binary comparisons, overlooking two key limitations: the diversity of human evaluators and the limitations of pairwise feedback. In this work, we address both these issues. First, we connect preference learning in RLHF with the econometrics literature and show that binary comparisons are insufficient for identifying latent user preferences from finite user data and infinite users, while (even incomplete) rankings over three or more responses ensure identifiability. Second, we introduce methods to incorporate heterogeneous preferences into alignment algorithms. We develop an Expectation-Maximization adaptation of DPO that discovers latent annotator types and trains a mixture of LLMs accordingly. Then we propose an aggregation algorithm using a min-max regret fairness criterion to produce a single generative policy with equitable performance guarantees. Together, these contributions establish a theoretical and algorithmic framework for fairness and personalization for diverse users in generative model alignment.
Semantic segmentation with coarse annotations
Venue: arXiv cs.CV 2025
Year: 2025

Semantic segmentation is the task of classifying each pixel in an image. Training a segmentation model achieves best results using annotated images, where each pixel is annotated with the corresponding class. When obtaining fine annotations is difficult or expensive, it may be possible to acquire coarse annotations, e.g. by roughly annotating pixels in an images leaving some pixels around the boundaries between classes unlabeled. Segmentation with coarse annotations is difficult, in particular when the objective is to optimize the alignment of boundaries between classes. This paper proposes a regularization method for models with an encoder-decoder architecture with superpixel based upsampling. It encourages the segmented pixels in the decoded image to be SLIC-superpixels, which are based on pixel color and position, independent of the segmentation annotation. The method is applied to FCN-16 fully convolutional network architecture and evaluated on the SUIM, Cityscapes, and PanNuke data sets. It is shown that the boundary recall improves significantly compared to state-of-the-art models when trained on coarse annotations.
Self-evolving expertise in complex non-verifiable subject domains: dialogue as implicit meta-RL
Venue: arXiv cs.AI 2025
Year: 2025

So-called `wicked problems', those involving complex multi-dimensional settings, non-verifiable outcomes, heterogeneous impacts and a lack of single objectively correct answers, have plagued humans throughout history. Modern examples include decisions over justice frameworks, solving environmental pollution, planning for pandemic resilience and food security. The use of state-of-the-art artificial intelligence systems (notably Large Language Model-based agents) collaborating with humans on solving such problems is being actively explored. While the abilities of LLMs can be improved by, for example, fine-tuning, hand-crafted system prompts and scaffolding with external tools, LLMs lack endogenous mechanisms to develop expertise through experience in such settings. This work address this gap with Dialectica, a framework where agents engage in structured dialogue on defined topics, augmented by memory, self-reflection, and policy-constrained context editing. Formally, discussion is viewed as an implicit meta-reinforcement learning process. The `dialogue-trained' agents are evaluated post-hoc using judged pairwise comparisons of elicited responses. Across two model architectures (locally run Qwen3:30b and OpenAI's o4-mini) results show that enabling reflection-based context editing during discussion produces agents which dominate their baseline counterparts on Elo scores, normalized Bradley-Terry-Davidson ability, and AlphaRank mass. The predicted signatures of learning are observed qualitatively in statement and reflection logs, where reflections identify weaknesses and reliably shape subsequent statements. Agreement between quantitative and qualitative evidence supports dialogue-driven context evolution as a practical path to targeted expertise amplification in open non-verifiable domains.
Demo: Guide-RAG: Evidence-Driven Corpus Curation for Retrieval-Augmented Generation in Long COVID
Venue: arXiv cs.AI 2025
Year: 2025

As AI chatbots gain adoption in clinical medicine, developing effective frameworks for complex, emerging diseases presents significant challenges. We developed and evaluated six Retrieval-Augmented Generation (RAG) corpus configurations for Long COVID (LC) clinical question answering, ranging from expert-curated sources to large-scale literature databases. Our evaluation employed an LLM-as-a-judge framework across faithfulness, relevance, and comprehensiveness metrics using LongCOVID-CQ, a novel dataset of expert-generated clinical questions. Our RAG corpus configuration combining clinical guidelines with high-quality systematic reviews consistently outperformed both narrow single-guideline approaches and large-scale literature databases. Our findings suggest that for emerging diseases, retrieval grounded in curated secondary reviews provides an optimal balance between narrow consensus documents and unfiltered primary literature, supporting clinical decision-making while avoiding information overload and oversimplified guidance. We propose Guide-RAG, a chatbot system and accompanying evaluation framework that integrates both curated expert knowledge and comprehensive literature databases to effectively answer LC clinical questions.
Learning Correlated Reward Models: Statistical Barriers and Opportunities
Venue: arXiv cs.LG 2025
Year: 2025

Random Utility Models (RUMs) are a classical framework for modeling user preferences and play a key role in reward modeling for Reinforcement Learning from Human Feedback (RLHF). However, a crucial shortcoming of many of these techniques is the Independence of Irrelevant Alternatives (IIA) assumption, which collapses \emph{all} human preferences to a universal underlying utility function, yielding a coarse approximation of the range of human preferences. On the other hand, statistical and computational guarantees for models avoiding this assumption are scarce. In this paper, we investigate the statistical and computational challenges of learning a \emph{correlated} probit model, a fundamental RUM that avoids the IIA assumption. First, we establish that the classical data collection paradigm of pairwise preference data is \emph{fundamentally insufficient} to learn correlational information, explaining the lack of statistical and computational guarantees in this setting. Next, we demonstrate that \emph{best-of-three} preference data provably overcomes these shortcomings, and devise a statistically and computationally efficient estimator with near-optimal performance. These results highlight the benefits of higher-order preference data in learning correlated utilities, allowing for more fine-grained modeling of human preferences. Finally, we validate these theoretical guarantees on several real-world datasets, demonstrating improved personalization of human preferences.
Memory-SAM: Human-Prompt-Free Tongue Segmentation via Retrieval-to-Prompt
Venue: arXiv cs.CV 2025
Year: 2025

Accurate tongue segmentation is crucial for reliable TCM analysis. Supervised models require large annotated datasets, while SAM-family models remain prompt-driven. We present Memory-SAM, a training-free, human-prompt-free pipeline that automatically generates effective prompts from a small memory of prior cases via dense DINOv3 features and FAISS retrieval. Given a query image, mask-constrained correspondences to the retrieved exemplar are distilled into foreground/background point prompts that guide SAM2 without manual clicks or model fine-tuning. We evaluate on 600 expert-annotated images (300 controlled, 300 in-the-wild). On the mixed test split, Memory-SAM achieves mIoU 0.9863, surpassing FCN (0.8188) and a detector-to-box SAM baseline (0.1839). On controlled data, ceiling effects above 0.98 make small differences less meaningful given annotation variability, while our method shows clear gains under real-world conditions. Results indicate that retrieval-to-prompt enables data-efficient, robust segmentation of irregular boundaries in tongue imaging. The code is publicly available at https://github.com/jw-chae/memory-sam.
SpeechLLMs for Large-scale Contextualized Zero-shot Slot Filling
Venue: arXiv cs.CL 2025
Year: 2025

Slot filling is a crucial subtask in spoken language understanding (SLU), traditionally implemented as a cascade of speech recognition followed by one or more natural language understanding (NLU) components. The recent advent of speech-based large language models (speechLLMs), which integrate speech and textual foundation models, has opened new avenues for achieving speech understanding tasks in a more unified, generative, and instruction-following manner while promising data and compute efficiency with zero-shot abilities, generalizing to unseen slot labels. We address the slot-filling task by creating an empirical upper bound for the task, identifying performance, robustness, and generalization gaps, and proposing improvements to the training data, architecture, and training strategies to narrow the gap with the upper bound result. We show that each of these measures improve performance substantially, while highlighting practical challenges and providing empirical guidance and insights for harnessing these emerging models.
BLIP3o-NEXT: Next Frontier of Native Image Generation
Venue: arXiv cs.CV 2025
Year: 2025

We present BLIP3o-NEXT, a fully open-source foundation model in the BLIP3 series that advances the next frontier of native image generation. BLIP3o-NEXT unifies text-to-image generation and image editing within a single architecture, demonstrating strong image generation and image editing capabilities. In developing the state-of-the-art native image generation model, we identify four key insights: (1) Most architectural choices yield comparable performance; an architecture can be deemed effective provided it scales efficiently and supports fast inference; (2) The successful application of reinforcement learning can further push the frontier of native image generation; (3) Image editing still remains a challenging task, yet instruction following and the consistency between generated and reference images can be significantly enhanced through post-training and data engine; (4) Data quality and scale continue to be decisive factors that determine the upper bound of model performance. Building upon these insights, BLIP3o-NEXT leverages an Autoregressive + Diffusion architecture in which an autoregressive model first generates discrete image tokens conditioned on multimodal inputs, whose hidden states are then used as conditioning signals for a diffusion model to generate high-fidelity images. This architecture integrates the reasoning strength and instruction following of autoregressive models with the fine-detail rendering ability of diffusion models, achieving a new level of coherence and realism. Extensive evaluations of various text-to-image and image-editing benchmarks show that BLIP3o-NEXT achieves superior performance over existing models.
Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery
Venue: arXiv cs.CV 2025
Year: 2025

Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task in providing immersive and embodied applications. The challenges lie in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by synergizing the readily available satellite imagery that supplies realistic coarse geometry and the open-domain diffusion model for creating high-quality close-up appearances. We propose \textbf{Skyfall-GS}, the first city-block scale 3D scene creation framework without costly 3D annotations, also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic textures. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/
Sketch-based Fluid Video Generation Using Motion-Guided Diffusion Models in Still Landscape Images
Venue: arXiv cs.GR 2025
Year: 2025

Integrating motion into static images not only enhances visual expressiveness but also creates a sense of immersion and temporal depth, establishing it as a longstanding and impactful theme in artistic expression. Fluid elements such as waterfall, river, and oceans are common features in landscape, but their complex dynamic characteristics pose significant challenges in modeling and controlling their motion within visual computing. Physics-based methods are often used in fluid animation to track particle movement. However, they are easily affected by boundary conditions. Recently, latent diffusion models have been applied to video generation tasks, demonstrating impressive capabilities in producing high-quality and temporally coherent results. However, it is challenging for the existing methods to animate fluid smooth and temporally consistent motion. To solve these issues, this paper introduces a framework for generating landscape videos by animating fluid in still images under the guidance of motion sketches. We propose a finetuned conditional latent diffusion model for generating motion field from user-provided sketches, which are subsequently integrated into a latent video diffusion model via a motion adapter to precisely control the fluid movement.
Opportunities and Challenges for 3D Systems and Their Design
Venue: arXiv cs.AR 2025
Year: 2025

Although it is not a new concept, 3D integration increasingly receives widespread interest and focus as lithographic scaling becomes more challenging, and as the ability to make miniature vias greatly improves. Like Moores law, 3D integration improves density. With improvements in packaging density, however, come the challenges associated with its inherently higher power density. And though it acts somewhat as a scaling accelerator, the vertical integration also poses new challenges to design and manufacturing technologies. The placement of circuits, vias, and macros in the planes of a 3D stack must be co-designed across layers (or must conform to new standards) so that, when assembled, they have correct spatial correspondence. Each layer, although perhaps being a mere functional slice through a system (and we can slice the system in many different ways), must be independently testable so that we can systematically test and diagnose subsystems before and after final assembly. When those layers are assembled, they must come together in a way that enables a sensible yield and facilitates testing the finished product. To make the most of 3D integration, we should articulate the leverages of 3D systems (other researchers offer a more complete treatment elsewhere). Then we can enumerate and elucidate many of the new challenges posed by the design, assembly, and test of 3D systems.
basic_RV32s: An Open-Source Microarchitectural Roadmap for RISC-V RV32I
Venue: arXiv cs.AR 2025
Year: 2025

This paper introduces BASIC_RV32s, an open-source framework providing a practical microarchitectural roadmap for the RISC-V RV32I architecture, addressing the gap between theoretical knowledge and hardware implementation. Following the classic Patterson and Hennessy methodology, the design evolves from a basic single-cycle core to a 5-stage pipelined core design with full hazard forwarding, dynamic branch prediction, and exception handling. For verification, the final core design is integrated into a System-on-Chip (SoC) with Universal Asynchronous Receiver-Transmitter (UART) communication implemented on a Xilinx Artix-7 Field-Programmable Gate Array (FPGA), achieving 1.09 Dhrystone million instructions per second per megahertz (DMIPS/MHz) at 50 MHz. By releasing all Register-Transfer Level (RTL) source code, signal-level logic block diagrams, and development logs under MIT license on GitHub, BASIC_RV32s offers a reproducible instructional pathway for the open-source hardware ecosystem.
Limited Read-Write/Set Hardware Transactional Memory without modifying the ISA or the Coherence Protocol
Venue: arXiv cs.AR 2025
Year: 2025

Hardware Transactional Memory (HTM) allows lock-free programming as easy as with traditional coarse-grain locks or similar, while benefiting from the performance advantages of fine-grained locking. Many HTM implementations have been proposed, but they have not received widespread adoption because of their high hardware complexity, their need for additions to the Instruction Set Architecture (ISA), and often for modifications to the cache coherence protocol. We show that HTM can be implemented without adding new instructions -- merely by extending the semantics of two existing, Load-Linked and Store-Conditional. Also, our proposed design does not modify or extend standard coherence protocols. We further propose to drastically simplify the implementation of HTM -- confined to modifications in the L1 Data Cache only -- by restricting it to applications where the write set plus the read set of each transaction do not exceed a small number of cache lines. We also propose two alternative mechanisms to guarantee forward progress, both based on detecting retrial attempts. We simulated our proposed design in Gem5, and we used it to implement several popular concurrent data structures, showing that a maximum of eight (8) words (cache lines) suffice for the write plus read sets. We provide a detailed explanation of selected implementations, clarifying the intended usage of our HTM from a programmer's perspective. We evaluated our HTM under varying contention levels to explore its scalability limits. The results indicate that our HTM provides good performance in concurrent data structures when contention is spread across multiple nodes: in such cases, the percentage of aborts relative to successful commits is very low. In the atomic fetch-and-increment benchmark for multiple shared counters, the results show that, under low-congestion, our HTM improves performance relative to the TTS lock.
Accelerating Frontier MoE Training with 3D Integrated Optics
Venue: arXiv cs.AR 2025
Year: 2025

The unabated growth in AI workload demands is driving the need for concerted advances in compute, memory, and interconnect performance. As traditional semiconductor scaling slows, high-speed interconnects have emerged as the new scaling engine, enabling the creation of larger logical GPUs by linking many GPUs into a single, low-latency, high-bandwidth compute domain. While initial scale-up fabrics leveraged copper interconnects for their power and cost advantages, the maximum reach of passive electrical interconnects (approximately 1 meter) effectively limits the scale-up domain to within a single rack. The advent of 3D-stacked optics and logic offers a transformative, power-efficient scale-up solution for connecting hundreds of GPU packages (thousands of GPUs) across multiple data center racks. This work explores the design tradeoffs of scale-up technologies and demonstrates how frontier LLMs necessitate novel photonic solutions to achieve aggressive power and performance targets. We model the benefits of 3D CPO (Passage) enabled GPUs and switches within the scale-up domain when training Frontier Mixture of Experts (MoE) models exceeding one trillion parameters. Our results show that the substantial increases in bandwidth and radix enabled by 3D CPO allow for an 8X increase in scale-up capability. This affords new opportunities for multi-dimensional parallelism within the scale-up domain and results in a 2.7X reduction in time-to-train, unlocking unprecedented model scaling.
SoCks - Simplifying Firmware and Software Integration for Heterogeneous SoCs
Venue: arXiv cs.AR 2025
Year: 2025

Modern heterogeneous System-on-Chip (SoC) devices integrate advanced components into a single package, offering powerful capabilities while also introducing significant complexity. To manage these sophisticated devices, firmware and software developers need powerful development tools. However, as these tools become increasingly complex, they often lack adequate support, resulting in a steep learning curve and challenging troubleshooting. To address this, this work introduces System-on-Chip blocks (SoCks), a flexible and expandable build framework that reduces complexity by partitioning the SoC image into high-level units called blocks. SoCks builds each firmware and software block in an encapsulated way, independently from other components of the image, thereby reducing dependencies to a minimum. While some information exchange between the blocks is unavoidable to ensure seamless runtime integration, this interaction is standardized via interfaces. A small number of dependencies and well-defined interfaces simplify the reuse of existing block implementations and facilitate seamless substitution between versions-for instance, when choosing root file systems for the embedded Linux operating system. Additionally, this approach facilitates the establishment of a decentralized and partially automated development flow through Continuous Integration and Continuous Delivery (CI/CD). Measurement results demonstrate that SoCks can build a complete SoC image up to three times faster than established tools.