Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,138 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
datasets:
|
| 4 |
+
- leduckhai/S-Chain
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
---
|
| 8 |
+
|
| 9 |
+
<p align="center">
|
| 10 |
+
<img src="./SChain_icon.png" alt="S-Chain logo" width="70">
|
| 11 |
+
</p>
|
| 12 |
+
|
| 13 |
+
<h1 align="center">S-Chain: Structured Visual Chain-of-Thought for Medicine</h1>
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
[](https://arxiv.org/abs/2510.22728)
|
| 18 |
+
[](https://huggingface.co/leduckhai/S-Chain)
|
| 19 |
+
[](https://huggingface.co/datasets/leduckhai/S-Chain)
|
| 20 |
+
[](https://github.com/leduckhai/S-Chain/blob/main/DATASET_LICENSE.md)
|
| 21 |
+
[](https://s-chain.github.io/)
|
| 22 |
+
[](https://github.com/leduckhai/S-Chain)
|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
⭐ **If you find this project helpful, please consider giving it a [star on GitHub](https://github.com/leduckhai/S-Chain)!**
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
|
| 30 |
+
<p align="center">
|
| 31 |
+
<a href="https://github.com/leduckhai" target="_blank"><strong>Khai Le-Duc</strong></a><sup>* 1,2✉</sup>,
|
| 32 |
+
<a href="https://scholar.google.com/citations?user=_NIyeykAAAAJ&hl=en" target="_blank"><strong>Duy M. H. Nguyen</strong></a><sup>* 3,4,24✉</sup>,
|
| 33 |
+
<a href="https://scholar.google.com/citations?user=5CbQH_kAAAAJ&hl=en" target="_blank"><strong>Phuong T. H. Trinh</strong></a><sup>* 5</sup>,
|
| 34 |
+
<strong>Tien-Phat Nguyen</strong><sup>* 6</sup>,
|
| 35 |
+
Nghiem T. Diep<sup>** 3</sup>,
|
| 36 |
+
An Ngo<sup>** 7</sup>,
|
| 37 |
+
Tung Vu<sup>** 8</sup>,
|
| 38 |
+
<a href="https://scholar.google.com/citations?user=trFdwLkAAAAJ&hl=en" target="_blank"><strong>Trinh Vuong</strong></a><sup>9</sup>,
|
| 39 |
+
Anh-Tien Nguyen<sup>10,11</sup>,
|
| 40 |
+
Mau Nguyen<sup>12</sup>,
|
| 41 |
+
Van Trung Hoang<sup>13</sup>,
|
| 42 |
+
<a href="https://scholar.google.com/citations?user=IMryD1YAAAAJ&hl=en" target="_blank"><strong>Khai-Nguyen Nguyen</strong></a><sup>14</sup>,
|
| 43 |
+
<a href="https://scholar.google.com/citations?user=ZAuQIqwAAAAJ&hl=en" target="_blank"><strong>Hy Nguyen</strong></a><sup>15</sup>,
|
| 44 |
+
Chris Ngo<sup>2</sup>,
|
| 45 |
+
<a href="https://scholar.google.com/citations?user=k_4zYecAAAAJ&hl=en" target="_blank"><strong>Anji Liu</strong></a><sup>16</sup>,
|
| 46 |
+
<a href="https://scholar.google.com/citations?user=Xs7cKMwAAAAJ&hl=en" target="_blank"><strong>Nhat Ho</strong></a><sup>17</sup>,
|
| 47 |
+
<a href="https://scholar.google.com/citations?user=Khifj_MAAAAJ&hl=en" target="_blank"><strong>Anne-Christin Hauschild</strong></a><sup>11</sup>,
|
| 48 |
+
<a href="https://scholar.google.com/citations?user=SmqouhIAAAAJ&hl=en" target="_blank"><strong>Khanh Xuan Nguyen</strong></a><sup>18</sup>,
|
| 49 |
+
<a href="https://scholar.google.com/citations?user=UrTlMiwAAAAJ&hl=en" target="_blank"><strong>Thanh Nguyen-Tang</strong></a><sup>19</sup>,
|
| 50 |
+
<a href="https://scholar.google.com/citations?user=cnncomYAAAAJ&hl=en" target="_blank"><strong>Pengtao Xie</strong></a><sup>20,21</sup>,
|
| 51 |
+
<a href="https://scholar.google.com/citations?user=v7i6Uz4AAAAJ&hl=en" target="_blank"><strong>Daniel Sonntag</strong></a><sup>3,22</sup>,
|
| 52 |
+
<a href="https://scholar.google.com/citations?user=23ZXZvEAAAAJ&hl=en" target="_blank"><strong>James Zou</strong></a><sup>23</sup>,
|
| 53 |
+
<a href="https://scholar.google.com/citations?user=p5vLzq0AAAAJ&hl=en" target="_blank"><strong>Mathias Niepert</strong></a><sup>4,24</sup>,
|
| 54 |
+
<a href="https://scholar.google.com/citations?user=EQw8d9AAAAAJ&hl=en" target="_blank"><strong>Anh Totti Nguyen</strong></a><sup>25✉</sup>
|
| 55 |
+
</p>
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
<p align="center">
|
| 59 |
+
<em>*Co-first authors; order randomized | **Co-second authors</em><br>
|
| 60 |
+
<em>✉ Corresponding Authors</em>
|
| 61 |
+
</p>
|
| 62 |
+
<details>
|
| 63 |
+
<summary><strong>🎓 Affiliations</strong> (click to expand)</summary>
|
| 64 |
+
1. University of Toronto, Canada
|
| 65 |
+
2. Knovel Engineering Lab, Singapore
|
| 66 |
+
3. German Research Centre for Artificial Intelligence
|
| 67 |
+
4. University of Stuttgart, Germany
|
| 68 |
+
5. Chonnam National University, South Korea
|
| 69 |
+
6. Singapore University of Technology and Design
|
| 70 |
+
7. Bucknell University, USA
|
| 71 |
+
8. Concordia University, Canada
|
| 72 |
+
9. Korea University
|
| 73 |
+
10. Justus Liebig University Giessen, Germany
|
| 74 |
+
11. University Medical Center Göttingen, Germany
|
| 75 |
+
12. Japan Advanced Institute of Science and Technology
|
| 76 |
+
13. Hue University, Vietnam
|
| 77 |
+
14. College of William & Mary, USA
|
| 78 |
+
15. Deakin University, Australia
|
| 79 |
+
16. National University of Singapore
|
| 80 |
+
17. University of Texas at Austin, USA
|
| 81 |
+
18. University of California, Berkeley, USA
|
| 82 |
+
19. New Jersey Institute of Technology, USA
|
| 83 |
+
20. University of California San Diego, USA
|
| 84 |
+
21. MBZUAI, UAE
|
| 85 |
+
22. Oldenburg University, Germany
|
| 86 |
+
23. Stanford University, USA
|
| 87 |
+
24. Max Planck Research School for Intelligent Systems (IMPRS-IS), Germany
|
| 88 |
+
25. Auburn University, USA
|
| 89 |
+
</details>
|
| 90 |
+
---
|
| 91 |
+
<p align="center">
|
| 92 |
+
✨ In honor of
|
| 93 |
+
<a href="https://en.wikipedia.org/wiki/H%E1%BA%A3i_Th%C6%B0%E1%BB%A3ng_L%C3%A3n_%C3%94ng" target="_blank"><strong>Hải Thượng Lãn Ông (海上懶翁) – Lê Hữu Trác (黎友晫)</strong></a>,
|
| 94 |
+
the father of Vietnamese traditional medicine ✨
|
| 95 |
+
</p>
|
| 96 |
+
## 🔍 What is S-Chain?
|
| 97 |
+
S-Chain is the first large-scale dataset of **Structured Visual Chain-of-Thought (SV-CoT)**:
|
| 98 |
+
each reasoning step is explicitly linked to visual evidence via bounding boxes.
|
| 99 |
+
This enables training and evaluating *grounded* medical VLM reasoning instead of
|
| 100 |
+
hallucinated justifications.
|
| 101 |
+
- **12,000 medical images** with expert bounding boxes.
|
| 102 |
+
- **700k+ VQA / rationale pairs** across **16 languages**.
|
| 103 |
+
- Each sample: image, question, answer, stepwise SV-CoT, and per-step visual regions.
|
| 104 |
+
|
| 105 |
+
We show that supervising VLMs with SV-CoT:
|
| 106 |
+
- Improves interpretability
|
| 107 |
+
- Improves grounding fidelity (reasoning actually points to the right region)
|
| 108 |
+
- Improves robustness across models and languages
|
| 109 |
+
|
| 110 |
+
<p align="center">
|
| 111 |
+
<img src="main_pipeline.png" alt="Alt text" width="1400"/>
|
| 112 |
+
</p>
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
## 📣 News
|
| 116 |
+
|
| 117 |
+
- **[Oct 2025]** Updated experiment scripts and checkpoints for ExGra-Med and LLaVA-Med. See the [readme](architectures/Exgra-Med-CoT/README.md) for detailed instructions.
|
| 118 |
+
- **[Oct 2025]** Dataset and project site released.
|
| 119 |
+
|
| 120 |
+
## Citation
|
| 121 |
+
If you find this work useful, please cite our paper: [https://arxiv.org/abs/2510.22728](https://arxiv.org/abs/2510.22728)
|
| 122 |
+
|
| 123 |
+
```
|
| 124 |
+
@article{leduc2025schain,
|
| 125 |
+
title={S-Chain: Structured Visual Chain-of-Thought For Medicine},
|
| 126 |
+
author={Le-Duc, Khai and Trinh, Phuong T. H. and Nguyen, Duy M. H. and Nguyen, Tien-Phat and Diep, Nghiem T. and Ngo, An and Vu, Tung and Vuong, Trinh and Nguyen, Anh-Tien and Nguyen, Mau and Hoang, Van Trung and Nguyen, Khai-Nguyen and Nguyen, Hy and Ngo, Chris and Liu, Anji and Ho, Nhat and Hauschild, Anne-Christin and Nguyen, Khanh Xuan and Nguyen-Tang, Thanh and Xie, Pengtao and Sonntag, Daniel and Zou, James and Niepert, Mathias and Nguyen, Anh Totti},
|
| 127 |
+
journal={arXiv preprint},
|
| 128 |
+
eprint={2510.22728},
|
| 129 |
+
url={https://arxiv.org/abs/2510.22728},
|
| 130 |
+
year={2025}
|
| 131 |
+
}
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
## ⚖️ Important Notice on Dataset Usage
|
| 135 |
+
|
| 136 |
+
The S-Chain dataset is provided solely for research and educational purposes.
|
| 137 |
+
It may contain human or machine annotation errors, as well as potential biases or inconsistencies inherent to medical data.
|
| 138 |
+
Users are expected to exercise appropriate caution in interpretation and ensure ethical and non-commercial use.
|