tool-research / index.html
evalstate's picture
evalstate HF Staff
Upload 4 files
6ba7af7 verified
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Tool Schema Rendering Atlas</title>
<meta
name="description"
content="Model-centric report showing how JSON tool definitions are rendered into prompts."
/>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=DM+Sans:ital,opsz,wght@0,9..40,300..800;1,9..40,300..800&family=Fraunces:ital,opsz,wght@0,9..144,300..900;1,9..144,300..900&family=Fira+Code:wght@400;600&display=swap"
rel="stylesheet"
/>
<link rel="stylesheet" href="./styles.css" />
</head>
<body>
<main class="app-shell">
<aside class="model-rail" aria-label="Models">
<div class="rail-head">
<p class="eyebrow">Tool Schema Rendering Atlas</p>
</div>
<button id="overview-link" class="overview-link" type="button">
<strong>Overview &amp; method</strong>
<span>What models see, and how this report was built</span>
</button>
<div class="legend" id="dialect-filter" aria-label="Dialect filter">
<button type="button" data-dialect="all" class="active">All</button>
<button type="button" data-dialect="typescript"><i class="dot typescript"></i>TypeScript</button>
<button type="button" data-dialect="xml"><i class="dot xml"></i>XML/tags</button>
<button type="button" data-dialect="json"><i class="dot json"></i>JSON</button>
<button type="button" data-dialect="custom-channel"><i class="dot custom-channel"></i>Custom</button>
<button type="button" data-dialect="message-json"><i class="dot message-json"></i>Message JSON</button>
</div>
<div id="model-list" class="model-list"></div>
</aside>
<section class="model-workspace">
<section id="overview-panel" class="overview-panel" hidden>
<div class="overview-hero">
<p class="eyebrow">What Do Models See?</p>
<h2>Tool schemas become prompt text.</h2>
<p>
This atlas compares what a model receives after a JSON tool definition has passed through tokenizer-aware chat templates, provider-style tool renderers, or model-supplied custom renderers.
</p>
</div>
<div class="method-diagram" aria-label="Methodology diagram">
<div class="method-step">
<span>1</span>
<strong>Tool definition</strong>
<p>Start from shared OpenAI-style JSON tool definitions covering required strings, nested arrays, enums, refs, nullable unions, and oneOf/allOf stress cases.</p>
</div>
<div class="method-arrow"></div>
<div class="method-step">
<span>2</span>
<strong>Tokenizer + template</strong>
<p>Download the model tokenizer where available, inspect its chat template and tool-use conventions, then render through realistic tokenizer templates or model-supplied custom renderer code.</p>
</div>
<div class="method-arrow"></div>
<div class="method-step">
<span>3</span>
<strong>Model-visible output</strong>
<p>Compare the rendered prompt text, feature survival, special modes, and copied evidence claims so each model’s tool dialect remains auditable.</p>
</div>
</div>
<div class="method-notes">
<section>
<p class="eyebrow">Reading The Report</p>
<h3>Start with a model, then compare the same input case across dialects.</h3>
<p>
The left rail groups models with identical sampled renderings, while the evidence section keeps each underlying packet visible. The right inspector records which schema features survived the template render.
</p>
</section>
<section>
<p class="eyebrow">Reading The Output</p>
<h3>The rendered prompt is shown whole.</h3>
<p>
Inline highlights call out tool names and schema-bearing keywords, but the report keeps the full rendered text visible so boilerplate, wrappers, and surrounding template instructions remain auditable.
</p>
</section>
</div>
<section class="exec-summary" id="exec-summary"></section>
</section>
<div id="matrix-view">
<header class="workspace-head">
<div>
<p class="eyebrow" id="active-family">Selected model</p>
<h2 id="active-model">Model</h2>
<p id="active-summary" class="summary-text"></p>
</div>
<div class="snapshot">
<div>
<span>Dialect</span>
<strong id="active-dialect"></strong>
</div>
<div>
<span>Support path</span>
<strong id="active-path"></strong>
</div>
<div>
<span>Coverage</span>
<strong id="active-coverage"></strong>
</div>
</div>
</header>
<section class="studio">
<div class="prompt-panel">
<div class="panel-head">
<div>
<h3 id="case-title">Same JSON tool schema, model-specific prompt dialect</h3>
</div>
<div class="case-picker">
<span>Example tool definition</span>
<div id="case-tabs" class="case-tabs" aria-label="Example tool definitions"></div>
</div>
</div>
<div class="view-tabs" role="tablist" aria-label="Transformation view">
<button id="input-tab" type="button" data-view="input" role="tab" aria-selected="false">
Tool Definition
</button>
<button id="output-tab" type="button" class="active" data-view="output" role="tab" aria-selected="true">
Model Visible Output
</button>
<div class="tab-tools">
</div>
</div>
<p class="render-caveat">
Template render evidence only. Hosted APIs or serving layers may normalize, reject, or rewrite tool schemas before the model receives them.
</p>
<pre id="rendered-output" class="code-pane rendered-code" tabindex="0"></pre>
<pre id="input-schema" class="code-pane input-code" tabindex="0" hidden></pre>
<div class="output-legend" aria-label="Inline token highlight legend">
<span><i class="swatch tool"></i>Tool names</span>
<span><i class="swatch schema"></i>Schema keywords</span>
<span><i class="swatch boilerplate"></i>Boilerplate text</span>
</div>
</div>
<aside class="inspector">
<div class="inspector-section">
<p class="eyebrow">Current transformation</p>
<p id="case-description" class="muted"></p>
</div>
<div class="inspector-section">
<p class="eyebrow">Feature survival</p>
<div id="feature-list" class="feature-list"></div>
</div>
<div class="inspector-section">
<p class="eyebrow">Special modes</p>
<div id="special-list" class="special-list"></div>
</div>
</aside>
</section>
<section class="evidence-packets">
<div class="panel-head">
<div>
<p class="eyebrow">Evidence packet</p>
<h3>Claims copied from each selected model’s findings JSON</h3>
</div>
</div>
<div id="claims" class="claims"></div>
</section>
</div>
</section>
</main>
<script src="./data.js"></script>
<script src="./app.js"></script>
</body>
</html>