docs/GUI_VISUALIZATION.md

Real-time GUI Visualization

Overview

The GUI visualization provides real-time, progressive updates as BA validation runs. It shows:

• 3D camera trajectories updating frame-by-frame
• Error metrics plotted as they're computed
• Statistics updating in real-time
• Progress indicators showing current status

Usage

Basic Usage

python scripts/run_arkit_ba_validation_gui.py \
    --arkit-dir assets/examples/ARKit \
    --output-dir data/arkit_ba_validation_gui \
    --max-frames 30 \
    --frame-interval 1 \
    --device cpu

GUI Components

Left Panel

1. Status Panel:

   • Current operation (e.g., "Running DA3 inference...")
   • Progress bar (indeterminate during processing)
   • Frame counter (e.g., "5/30 frames")

2. Statistics Panel:

   • Real-time statistics updating as data arrives
   • Mean/max rotation errors
   • Mean translation errors
   • Comparison metrics (DA3 vs ARKit, BA vs ARKit, DA3 vs BA)

Right Panel (Tabs)

1. 3D Trajectories Tab:

   • Interactive 3D plot showing camera paths
   • Green: ARKit (ground truth)
   • Red: DA3 predictions
   • Blue: BA refined poses
   • Updates progressively as frames are processed
   • Can rotate, zoom, pan using matplotlib toolbar

2. Error Metrics Tab:

   • Rotation errors plotted over time
   • Threshold lines (2° accept, 30° reject)
   • Updates as errors are computed

3. Comparison Tab:

   • Side-by-side comparison of rotation and translation errors
   • Multiple methods shown together
   • Updates progressively

Progressive Updates

The GUI updates in real-time as:

1. Frame Extraction: Progress bar shows frame extraction
2. DA3 Inference:
   • Status updates to "Running DA3 inference..."
   • Trajectories update as each frame's pose is computed
   • Progress counter increments
3. BA Validation:
   • Status updates to "Running BA validation..."
   • BA poses appear on trajectory plot
   • Error metrics update as computed
4. Completion:
   • Final statistics displayed
   • All visualizations finalized

Thread Safety

The GUI uses a thread-safe update mechanism:

• Validation runs in a background thread
• Updates are queued and processed in the main GUI thread
• No blocking of the GUI during computation

Features

Real-time Updates

• Visualizations update as data arrives
• No need to wait for entire process to complete
• See results immediately

Interactive 3D Plot

• Rotate, zoom, pan using matplotlib toolbar
• Toggle trajectories on/off (via legend)
• Export to image (via toolbar)

Error Analysis

• See error patterns as they develop
• Identify problematic frames early
• Compare methods side-by-side

Statistics Panel

• Real-time statistics
• Scrollable text area
• Auto-updates as new data arrives

Example Workflow

1. Start GUI: Run the script, GUI window opens
2. Watch Progress:
   • Status shows "Processing ARKit data..."
   • Progress bar animates
3. DA3 Inference:
   • Trajectory plot shows ARKit path (green)
   • DA3 poses appear (red) as computed
   • Statistics update
4. BA Validation:
   • BA poses appear (blue)
   • Error metrics populate
   • Final statistics displayed
5. Analysis:
   • Rotate 3D plot to inspect trajectories
   • Check error plots for patterns
   • Review statistics panel

Troubleshooting

GUI Not Updating

• Check that validation thread is running (status should change)
• Verify no exceptions in console
• Ensure GUI main loop is running (window should be responsive)

Performance Issues

• Reduce --max-frames for faster updates
• Use --device cpu if GPU is slow
• Increase frame_interval to process fewer frames

Missing Updates

• Some updates may be batched for performance
• Check statistics panel for final results
• All data is available after completion

Integration

The GUI can be integrated into other workflows:

from ylff.visualization_gui import create_gui
from scripts.run_arkit_ba_validation_gui import run_validation_with_gui

# Create GUI
gui = create_gui()

# Start validation
run_validation_with_gui(
    gui,
    arkit_dir=Path("path/to/arkit"),
    output_dir=Path("output"),
    max_frames=30,
)

# GUI runs until window is closed

Advantages Over Static Visualization

1. Immediate Feedback: See results as they're computed
2. Early Problem Detection: Identify issues before completion
3. Interactive Exploration: Rotate/zoom 3D plots in real-time
4. Progress Monitoring: Know exactly what's happening
5. No Post-Processing: Results available immediately