Update README.md

README.md

@@ -1,3 +1,155 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+datasets:
+- internlm/VC-RewardBench
+base_model:
+- Qwen/Qwen3-VL-8B-Instruct
+---
+
+<p align="center">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/63859cf3b2906edaf83af9f0/gcuIXKMoDd-nQoPrynVQF.png" width="30%">
+</p>
+
+# Visual-ERM
+
+Visual-ERM is a **multimodal generative reward model** for **vision-to-code** tasks.  
+It evaluates outputs directly in the **rendered visual space** and produces **fine-grained**, **interpretable**, and **task-agnostic** discrepancy feedback for structured visual reconstruction.
+
+<p align="center">
+  <a href="https://arxiv.org/abs/2603.13224">📄 Paper</a> |
+  <a href="https://github.com/InternLM/Visual-ERM">💻 GitHub</a> |
+  <a href="https://huggingface.co/datasets/internlm/VC-RewardBench">📊 VC-RewardBench</a>
+</p>
+
+## Model Overview
+
+Existing rewards for vision-to-code usually fall into two categories:
+
+1. **Text-based rewards** such as edit distance or TEDS, which ignore important visual cues like layout, spacing, alignment, and style.
+2. **Vision embedding rewards** such as DINO similarity, which are often coarse-grained and can be vulnerable to reward hacking.
+
+Visual-ERM addresses this by directly comparing:
+
+- the **ground-truth image**, and
+- the **rendered image** produced from a model prediction,
+
+and then generating **structured discrepancy annotations** that can be converted into reward signals or used for reflection-based refinement.
+
+## What this model does
+
+Visual-ERM is designed to judge whether a predicted result is **visually equivalent** to the target.
+
+Given a pair of images, it can identify discrepancies such as:
+
+- **category**
+- **severity**
+- **location**
+- **description**
+
+This makes Visual-ERM useful not only as a reward model for RL, but also as a **visual critic** for test-time reflection and revision.
+
+## Supported Tasks
+
+Visual-ERM is designed for structured visual reconstruction tasks, including:
+
+- **Chart-to-Code**
+- **Table-to-Markdown**
+- **SVG-to-Code**
+
+## Key Features
+
+- **Visual-space reward modeling**  
+  Evaluates predictions in rendered visual space instead of relying only on text matching or coarse embedding similarity.
+
+- **Fine-grained and interpretable feedback**  
+  Produces structured discrepancy annotations rather than a single black-box score.
+
+- **Task-agnostic reward supervision**  
+  A unified reward model that generalizes across multiple vision-to-code tasks.
+
+- **Useful for both training and inference**  
+  Can be used as a reward model in RL and as a visual critic during test-time refinement.
+
+## VC-RewardBench
+
+We also release **VisualCritic-RewardBench (VC-RewardBench)**, a benchmark for evaluating fine-grained image-to-image discrepancy judgment on structured visual data.
+
+### Benchmark Features
+
+- Covers **charts**, **tables**, and **SVGs**
+- Contains **1,335** carefully curated instances
+- Each instance includes:
+  - a ground-truth image
+  - a corrupted / rendered counterpart
+  - fine-grained discrepancy annotations
+
+Dataset link:  
+https://huggingface.co/datasets/internlm/VC-RewardBench
+
+## How to Use
+
+Visual-ERM is fine-tuned from **Qwen/Qwen3-VL-8B-Instruct** and follows the same multimodal interface.
+
+### Input
+
+Visual-ERM takes as input:
+
+- a **reference / ground-truth image**
+- a **rendered prediction image**
+- a **prompt** asking the model to identify fine-grained visual discrepancies
+
+### Output
+
+The model outputs structured discrepancy annotations, which can then be:
+
+- converted into a scalar reward for RL
+- used as feedback for reflection-and-revision
+- evaluated directly on VC-RewardBench
+
+A typical output format is:
+
+```json
+{
+  "errors": [
+    {
+      "category": "structure_error",
+      "severity": 3,
+      "location": "legend area",
+      "description": "The legend is placed outside the plot area in the prediction."
+    },
+    {
+      "category": "style_error",
+      "severity": 2,
+      "location": "bar colors",
+      "description": "The colors differ from those in the reference image."
+    }
+  ]
+}
+```
+
+### Inference / Evaluation / RL
+
+For full inference scripts, RL training pipelines, evaluation code, and prompt templates, please refer to the official repository:
+
+https://github.com/InternLM/Visual-ERM
+
+## Intended Use
+
+Visual-ERM is intended for:
+
+- **reward modeling** in vision-to-code RL pipelines
+- **visual discrepancy judgment** between target and predicted renderings
+- **reflection-based refinement** at inference time
+- **research on visual reward modeling** and multimodal RL
+
+## Citation
+
+If you find this model useful, please consider citing:
+
+```bibtex
+TBD
+```
+
+## Contact
+
+If you are interested in **visual reward modeling**, **vision-to-code**, or **reinforcement learning for multimodal models**, feel free to reach out.