Upload 10-episode all-metrics verification sample with reproduction files
Browse files
artifacts/results/metric_iter31_sample10_all_metrics_verify/README.md
CHANGED
|
@@ -5,3 +5,48 @@ This folder contains 10 randomly selected episodes rendered in `all_metrics`-onl
|
|
| 5 |
Selection seed: `20260408`
|
| 6 |
|
| 7 |
Episodes: `episode97`, `episode90`, `episode73`, `episode69`, `episode12`, `episode63`, `episode47`, `episode43`, `episode14`, `episode46`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
Selection seed: `20260408`
|
| 6 |
|
| 7 |
Episodes: `episode97`, `episode90`, `episode73`, `episode69`, `episode12`, `episode63`, `episode47`, `episode43`, `episode14`, `episode46`
|
| 8 |
+
|
| 9 |
+
## What Is Included
|
| 10 |
+
|
| 11 |
+
For each sampled episode directory:
|
| 12 |
+
|
| 13 |
+
- `episodeX.dense.csv`: dense metric table from the completed 100-episode single-pass fuller logging run
|
| 14 |
+
- `episodeX.keyframes.csv`: keyframe downsample for the same episode
|
| 15 |
+
- `episodeX.metrics.json`: per-episode summary metrics
|
| 16 |
+
- `episodeX.debug.jsonl`: fuller per-frame debug sidecar
|
| 17 |
+
- `templates.json` and `templates.pkl`: fixed template bundle used for replay and rendering
|
| 18 |
+
- `visualizations/episodeX_all_metrics.gif`: compact verification GIF
|
| 19 |
+
- `visualizations/README.md`: per-episode visualization note
|
| 20 |
+
|
| 21 |
+
This sample bundle was derived from the completed 100-episode run:
|
| 22 |
+
|
| 23 |
+
- `metric_iter30_full100_single_pass_full_logging_fixed_templates_merged`
|
| 24 |
+
|
| 25 |
+
## Render Mode
|
| 26 |
+
|
| 27 |
+
These GIFs were rendered with the fast `all_metrics`-only path added in:
|
| 28 |
+
|
| 29 |
+
- `code/scripts/render_oven_metric_frame.py`
|
| 30 |
+
- `code/scripts/render_oven_metric_gifs.py`
|
| 31 |
+
|
| 32 |
+
That mode skips simulator replay when only the full-metrics panel is requested, so the render depends only on:
|
| 33 |
+
|
| 34 |
+
- stored RGB frames in the RLBench episode directory
|
| 35 |
+
- the dense CSV row for each rendered frame
|
| 36 |
+
|
| 37 |
+
## Reproduce
|
| 38 |
+
|
| 39 |
+
Per episode, the render command is:
|
| 40 |
+
|
| 41 |
+
```bash
|
| 42 |
+
python code/scripts/render_oven_metric_gifs.py \
|
| 43 |
+
--episode-dir /workspace/data/bimanual_take_tray_out_of_oven_train_128/all_variations/episodes/episode97 \
|
| 44 |
+
--dense-csv artifacts/results/metric_iter31_sample10_all_metrics_verify/episode97/episode97.dense.csv \
|
| 45 |
+
--templates-pkl artifacts/results/metric_iter31_sample10_all_metrics_verify/episode97/templates.pkl \
|
| 46 |
+
--output-dir artifacts/results/metric_iter31_sample10_all_metrics_verify/episode97/visualizations \
|
| 47 |
+
--all-metrics-only \
|
| 48 |
+
--num-workers 8 \
|
| 49 |
+
--base-display 1500
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
The sampled episode list and seed are also recorded in `selection.json`.
|
artifacts/results/metric_iter31_sample10_all_metrics_verify/selection.json
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
{
|
| 2 |
"type": "sample_verification_bundle",
|
|
|
|
| 3 |
"source_result_root": "/workspace/VLAdaptorBench_upload/artifacts/results/metric_iter30_full100_single_pass_full_logging_fixed_templates_merged",
|
| 4 |
"selection_seed": 20260408,
|
| 5 |
"episode_indices": [
|
|
@@ -16,4 +17,4 @@
|
|
| 16 |
],
|
| 17 |
"render_mode": "all_metrics_only",
|
| 18 |
"created_at_epoch": 1775672903.333171
|
| 19 |
-
}
|
|
|
|
| 1 |
{
|
| 2 |
"type": "sample_verification_bundle",
|
| 3 |
+
"source_run_name": "metric_iter30_full100_single_pass_full_logging_fixed_templates_merged",
|
| 4 |
"source_result_root": "/workspace/VLAdaptorBench_upload/artifacts/results/metric_iter30_full100_single_pass_full_logging_fixed_templates_merged",
|
| 5 |
"selection_seed": 20260408,
|
| 6 |
"episode_indices": [
|
|
|
|
| 17 |
],
|
| 18 |
"render_mode": "all_metrics_only",
|
| 19 |
"created_at_epoch": 1775672903.333171
|
| 20 |
+
}
|