Omkar2118/datasets huggingface dataset:
StableToolBench

StableToolBench is a new benchmark for tool learning that aims to provide a well-balanced combination of stability and reality, building upon its predecessor, ToolBench. It was developed to address the instability issues of previous tool learning benchmarks, which either relied on hand-crafted online tools with limited scale or large-scale real online APIs that suffered from instability due to API status changes¹².

Here are some key features of StableToolBench:
- **Virtual API System**: This includes a caching system to ensure consistent API call responses and API simulators, powered by Large Language Models (LLMs), for unavailable APIs. It maintains the diverse API environment from ToolBench¹.
- **New Set of Solvable Queries**: It uses state-of-the-art LLMs to determine task solvability beforehand, reducing randomness and instability in query solvability¹.
- **Stable Evaluation System**: Implements a two-phase evaluation process using GPT-4 as an automatic evaluator, with metrics like Solvable Pass Rate (SoPR) and Solvable Win Rate (SoWR) to assess the capability of LLMs to utilize tools².

(1) THUNLP-MT/StableToolBench - GitHub. https://github.com/THUNLP-MT/StableToolBench.
(2) StableToolBench: Towards Stable Large-Scale Benchmarking on Tool .... https://arxiv.org/abs/2403.07714.
(3) StableToolBench: Towards Stable Large-Scale Benchmarking on Tool .... https://paperreading.club/page?id=214642.
(4) Papers with Code - StableToolBench: Towards Stable Large-Scale .... https://paperswithcode.com/paper/stabletoolbench-towards-stable-large-scale.
(5) undefined. https://doi.org/10.48550/arXiv.2403.07714.

Introducing paper: StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models
AgentEval

AgentEval is part of the AgentGym framework, which is designed to evaluate and develop generally-capable Large Language Model-based (LLM-based) agents. AgentEval serves as a benchmark suite within AgentGym, providing a set of tasks and environments to assess the performance of these agents¹².

The AgentGym framework includes diverse interactive environments and tasks with a unified format, supporting real-time feedback and concurrency, which is essential for the development and scaling of LLM-based agents. The benchmark suite, AgentEval, along with the trajectory sets AgentTraj and AgentTraj-L, enables researchers and developers to measure the capabilities of agents across a broad spectrum of tasks and environments².

(1) AgentGym: Evolving Large Language Model-based Agents across Diverse .... https://arxiv.org/html/2406.04151v1.
(2) GitHub - WooooDyy/AgentGym: Code and implementations for the paper .... https://github.com/WooooDyy/AgentGym.
(3) AgentGym: Evolving Large Language Model-based Agents across Diverse .... https://agentgym.github.io/.
(4) undefined. https://agentgym.github.io.

Introducing paper: AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
National Lung Screening Trial (NLST)
Full name: National Lung Screening Trial (NLST)

The National Lung Screening Trial (NLST) was a randomized controlled trial conducted by the Lung Screening Study group (LSS) and the American College of Radiology Imaging Network (ACRIN) to determine whether screening for lung cancer with low-dose helical computed tomography (CT) reduces mortality from lung cancer in high-risk individuals relative to screening with chest radiography. Approximately 54,000 participants were enrolled between August 2002 and April 2004. Data collection has ended, and information is complete through December 31, 2009. NLST has the ClinicalTrials.gov registration number NCT00047385.