Update README.md

README.md

@@ -2,6 +2,17 @@
 title: >-
   BioProBench: Comprehensive Dataset and Benchmark in Biological Protocol
   Understanding and Reasoning
+license: cc-by-4.0
+task_categories:
+  - text-generation
+  - question-answering
+language:
+  - en
+tags:
+  - biology
+  - protocol
+  - benchmark
+  - ai4science
 ---
 
 <div align="center">
@@ -12,98 +23,77 @@ title: >-
 
 [![ArXiv](https://img.shields.io/badge/ArXiv-paper-B31B1B.svg?logo=arXiv&logoColor=Red)](https://arxiv.org/pdf/2505.07889)
 [![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Dataset-FFD210.svg?logo=HuggingFace&logoColor=black)](https://huggingface.co/BioProBench)
+[![GitHub](https://img.shields.io/badge/GitHub-Code-181717.svg?logo=github&logoColor=white)](https://github.com/YuyangSunshine/bioprotocolbench)
+[![Project Page](https://img.shields.io/badge/Project-Page-blue.svg?logo=google-chrome&logoColor=white)](https://yuyangsunshine.github.io/BioPro-Project/)
 [![License: CC BY 4.0](https://img.shields.io/badge/License-CC_BY_4.0-lightgrey.svg)](https://creativecommons.org/licenses/by/4.0/)
-[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/yourusername/bioprotocolbench/pulls)
 
-**BioProBench is the first large-scale, integrated multi-task benchmark for biological protocol understanding and reasoning, specifically designed for large language models (LLMs). It moves beyond simple QA to encompass a comprehensive suite of tasks critical for procedural text comprehension.**
+---
+
+## 📢 Latest News 
+* ✨ **[2026-03-31]** **Data Split Update!** We have officially released the **Train/Test splits** for each task (PQA, ORD, ERR, GEN, REA), making it easier for the community to train and evaluate models consistently.
+* 🔥 **[2026-03-18]** Our **BioProAgent** is now live on AI4S LAB! [Try it out and order wet-lab experiments here](https://yuyangsunshine.github.io/BioPro-Project/).
+* 🎉 **[2026-03-03]** Our BioProAgent has been accepted by the **ICLR 2026 LLA Workshop!**
+* 📄 **[2026-03-01]** The preprint of our **BioProAgent** paper is available on [arXiv](https://arxiv.org/pdf/2505.07889).
+* 📝 **[2026-01-21]** BioProBench paper has been updated with new experimental results.
+* 🚀 **[2025-12-01]** Code and dataset (v1.0) are released on GitHub.
 
 ---
 
-Biological protocols are the fundamental bedrock of reproducible and safe life science research. While LLMs have shown remarkable capabilities on general tasks, their systematic evaluation on highly specialized, accuracy-critical, and inherently procedural texts like biological protocols remains limited. BioProBench fills this gap by providing a robust framework to evaluate LLMs on diverse aspects of protocol understanding and reasoning.
+## 🌟 Introduction
+**BioProBench** is the first large-scale, integrated multi-task benchmark for biological protocol understanding and reasoning, specifically designed for Large Language Models (LLMs). It moves beyond simple QA to encompass a comprehensive suite of tasks critical for procedural text comprehension in life sciences.
 
 <div align="center">
-  <img src="https://github.com/YuyangSunshine/bioprotocolbench/blob/main/figures/overview.png?raw=true" alt="BioProBench Logo" width="1000"/>
+  <img src="https://github.com/YuyangSunshine/bioprotocolbench/blob/main/figures/overview.png?raw=true" alt="BioProBench Overview" width="1000"/>
 </div>
 
-BioProBench features:
+### Key Features:
 * 📚 **Large-scale Data:** Built upon **27K original biological protocols**, yielding nearly **556K high-quality structured instances**.
-* 🎯 **Comprehensive Tasks:** A suite of ** five core tasks** challenging LLMs on different facets of procedural understanding and generation:
-
-    * Protocol Question Answering (PQA)
-    * Step Ordering (ORD)
-    * Error Correction (ERR)
-    * Protocol Generation (GEN)
-    * Protocol Reasoning (REA)
-* 🧬 **Broad Domain Coverage:** Data sourced from 6 major repositories and covering **16 biological subdomains**.
-* 🔬 **Standardized Evaluation:** A robust framework combining standard NLP metrics with novel domain-specific measures for accurate performance quantification.
-  
----
-
-
-## 🚀 Motivation
-
-Biological protocols are the operational blueprint for experiments. As biological research increasingly leverages automation and AI, the ability of AI systems to understand and reason about these complex procedures is paramount. Current LLMs, while powerful, face significant challenges:
-
-* Limited Procedural Understanding:
-* LLMs struggle with the temporal dependencies, conditional logic, and specific requirements embedded within protocols.
-* Lack of Systematic Evaluation: There has been a lack of large-scale, multi-task benchmarks specifically designed to diagnose LLMs' limitations on procedural biological texts.
-* Bridging the Gap: Developing AI systems capable of safely automating and even optimizing experiments requires models that can reliably interpret and generate protocols.
-
-BioProBench addresses these challenges by providing the necessary data and tasks for comprehensive evaluation and driving the development of more capable models.
+* 🎯 **Comprehensive Tasks:** 5 core tasks: **PQA** (Question Answering), **ORD** (Step Ordering), **ERR** (Error Correction), **GEN** (Generation), and **REA** (Reasoning).
+* 🧬 **Broad Domain Coverage:** Covers **16 biological subdomains** from 6 major repositories.
+* 🔬 **Standardized Evaluation:** A robust framework combining NLP metrics with novel domain-specific measures.
 
 ---
 
-## 📊 Dataset Structure
+## 📊 Dataset Structure & Tasks
 
 <div align="center">
-  <img src="https://github.com/YuyangSunshine/bioprotocolbench/blob/main/figures/samples.jpg?raw=true" alt="BioProBench Logo" width="1000"/>
+  <img src="https://github.com/YuyangSunshine/bioprotocolbench/blob/main/figures/samples.jpg?raw=true" alt="BioProBench Samples" width="1000"/>
 </div>
 
-BioProBench provides a layered data design to support various model development stages:
+We provide standardized JSON files for each task, now including **Train** and **Test** splits:
 
-* A raw corpus of **27K protocols** for pretraining or RAG applications.
-* A substantial downstream **training set of over 550K structured instances** across the five fine-grained tasks for model adaptation.
-* A held-out **test set of 1,000 examples per task** for standardized benchmarking.
+| Task | Description | Files |
+| :--- | :--- | :--- |
+| **PQA** | Protocol Question Answering | `PQA_train.json`, `PQA_test.json` |
+| **ORD** | Step Ordering | `ORD_train.json`, `ORD_test.json` |
+| **ERR** | Error Correction | `ERR_train.json`, `ERR_test.json` |
+| **GEN** | Protocol Generation | `GEN_train.json`, `GEN_test.json` |
+| **Raw** | Full Protocol Corpus | `Bio-protocol.json`, `Protocol-io.json`, etc. |
 
-The dataset and code are publicly available:
-
-* **Code Repository:** [https://github.com/YuyangSunshine/bioprotocolbench](https://github.com/YuyangSunshine/bioprotocolbench/)
-* **Hugging Face Dataset:** [https://huggingface.co/BioProBench](https://huggingface.co/BioProBench)
+### 🔗 Useful Links
+* **Official Website:** [BioPro-Project Page](https://yuyangsunshine.github.io/BioPro-Project/)
+* **GitHub Repository:** [bioprotocolbench](https://github.com/YuyangSunshine/bioprotocolbench) (Code for evaluation & training)
 
 ---
 
-#### 🔬 Key Findings
-We evaluated 12 mainstream open-source and closed-source LLMs on BioProBench. Our key findings reveal significant insights into the current capabilities and limitations of LLMs on biological protocols:
-
-* Surface vs. Deep Understanding: While top models perform reasonably well on tasks requiring surface understanding like Protocol Question Answering (e.g., ~70% PQA-Acc.) and Error Correction (e.g., >64% ERR F1), they struggle significantly with tasks demanding deeper procedural understanding and structured generation.
-* Challenges in Reasoning and Generation: Performance drops considerably on Step Ordering (e.g., ORD-EM ~50%) and Protocol Generation (e.g., GEN-BLEU <15%), highlighting the difficulty for LLMs in managing temporal dependencies and generating coherent, accurate procedures.
-* Model Variances: Comparisons show diverse performance across models. Certain open-source models approach the performance of closed-source models on some tasks, while smaller, bio-specific models often lag behind general LLMs on complex procedural content, suggesting limitations in capacity for capturing intricate dependencies.
-
-Overall, our findings underscore that robust procedural reasoning within biological protocols represents a significant challenge for current LLMs.
+## 🔬 Key Findings
+We evaluated 12 mainstream LLMs. Our findings reveal:
+* **Surface vs. Deep Understanding:** Models perform well on QA (~70% Acc) but struggle with deep procedural logic.
+* **Reasoning Bottleneck:** Performance drops significantly on **Step Ordering** and **Protocol Generation** (BLEU < 15%), highlighting the difficulty of managing temporal dependencies.
+* **Bio-specific Models:** Interestingly, some bio-specific models lag behind general LLMs in capturing intricate procedural dependencies, suggesting a need for larger reasoning capacity.
 
 ---
 
-## 🤝 Contributing
-We welcome contributions to enhance BioProBench, including:
-  - New protocol sources
-  - 🧪 Additional biological domains
-  - 🧠 Novel evaluation tasks
-  - 📝 Annotation improvements
-
----
+## 🤝 Contributing & Contact
+We welcome contributions such as new protocol sources, additional domains, or novel tasks!
+- **Email:** sunshineliuyuyang@gmail.com
+- **Issues:** Feel free to open an issue on our [GitHub](https://github.com/YuyangSunshine/bioprotocolbench).
 
 ## 📜 Citation
 ```bibtex
 @misc{bioprotocolbench2025,
   title={BioProBench: Comprehensive Dataset and Benchmark in Biological Protocol Understanding and Reasoning},
-  author={Yuyang Liu⋆, Liuzhenghao Lv⋆ Xiancheng Zhang, Jingya Wang, Li Yuan1, Yonghong Tian.},
+  author={Yuyang Liu, Liuzhenghao Lv, Xiancheng Zhang, Jingya Wang, Li Yuan, Yonghong Tian},
   year={2025},
-  url={https://arxiv.org/pdf/2505.07889}
-}
-```
-
----
-
-## 📧 Contact
-For dataset access or collaboration inquiries:
-sunshineliuyuyang@gmail.com
+  url={[https://arxiv.org/pdf/2505.07889](https://arxiv.org/pdf/2505.07889)}
+}