ebadhussain20/yolov11s_crowd

HF Evaluation Platform

A unified Hugging Face model benchmarking + license/governance review dashboard, built by merging two existing repositories into one modular Streamlit app:

• model_license_reviewer — a Kaggle notebook (HF-01.06) that reviews license, commercial-use terms, attribution, privacy risk, and approves/rejects a model.
• model_benchmark — a Python package that benchmarks latency, throughput, memory, and GPU/CPU usage for a HF model.

Task: HF-01.07 — Build the Evaluation Dashboard.

What this delivers against HF-01.07

Acceptance criterion	Status
Dashboard loads registry data	✅ RegistryService merges benchmark + license outputs
Filters work correctly (model, task, hardware, license, performance, accuracy)	✅ Model Registry page
Exports valid files (CSV, JSON)	✅ every page has CSV/JSON download buttons; Markdown export also included
Shows benchmark charts	✅ latency/throughput/memory/accuracy/perf charts (Plotly)
Shows recommendation notes	✅ per-model notes + a rule-based Recommendation Engine page
Evidence: screenshots, exports, README, tests, demo	✅ exports/ has sample CSV/JSON/Markdown; tests/ has 17 passing tests; see "Running the app" for the live demo

Important note on data — read this first

This environment has no network access to huggingface.co and no GPU, so the live pipelines in both source repos (which download real models and call the Hugging Face Hub API) cannot run here.

To keep the platform honest and fully working end-to-end, RegistryService runs in one of two modes:

1. Live mode — set HF_EVAL_BENCHMARK_CSV and HF_EVAL_LICENSE_CSV to real report.csv / model_license_review.csv files produced by running model_benchmark and model_license_reviewer yourself (e.g. on your RTX 3090 box or a Kaggle GPU notebook). The platform will merge and display that real data — no code changes needed.
2. Sample mode (default) — if those env vars aren't set, RegistryService generates a realistic 20-model sample dataset. Critically, the license/governance classification is not faked: the sample data is run through the actual extracted governance functions (parse_license, classify_commercial_use, analyze_privacy_risk, decide_approval, etc. in app/license_review/reviewer.py, lifted directly from the notebook) — only the raw model-card text and benchmark numbers are synthetic. So the approval/rejection logic you see in the dashboard is identical to what the real pipeline would produce given the same inputs.

Architecture

hf-evaluation-platform/
  run.py                        # Streamlit entry point (Home/Overview)
  pages/                        # Streamlit multipage dashboard
    1_Model_Registry.py         # search, filter, sort, pagination, export
    2_Benchmark_Dashboard.py    # latency/throughput/memory/accuracy charts, ranking
    3_License_Dashboard.py      # per-model governance review
    4_Comparison.py             # multi-model comparison + radar chart
    5_Recommendations.py        # rule-based recommendation engine
    6_Analytics.py              # distribution charts, full export
  app/
    config/settings.py          # env-var driven configuration
    license_review/reviewer.py  # extracted from model_license_reviewer notebook (HF-01.06)
    services/
      registry_service.py       # merges benchmark + license data (real or sample)
      recommendation_service.py # deterministic, explainable recommendations
      chart_service.py          # Plotly figure builders
      export_service.py         # CSV / JSON / Markdown export
  data/                         # generated: registry.csv, benchmark_results.csv, license_review.csv
  exports/                      # sample export evidence (see below)
  tests/                        # pytest unit tests (17 passing)

No components from model_benchmark were rewritten — it's a mature, well-structured package (benchmark_runner.py, reporters/, charts/, hardware.py, etc.) and its CSV/JSON report schema (report.csv columns: model_id, task, device, mean_latency_ms, throughput_req_per_sec, peak_ram_mb, peak_gpu_mb, ...) is consumed as-is by RegistryService.load_registry(). The license reviewer's logic (license parsing, commercial-use rules, privacy-risk keyword analysis, the approval engine's rejection rules) was extracted unchanged from the notebook into a proper importable module so it can be unit tested and reused instead of copy-pasted.

Installation

cd hf-evaluation-platform
pip install -r requirements.txt

Usage

streamlit run run.py

Then open the printed local URL. Use the sidebar in Model Registry to filter by model, task, license, hardware, approval, latency range, and accuracy range; use the download buttons on any page to export CSV/JSON (and Markdown on the Recommendations page).

To point at real data from your own benchmark/review runs:

export HF_EVAL_BENCHMARK_CSV=/path/to/model_benchmark/examples/example_report/report.csv
export HF_EVAL_LICENSE_CSV=/path/to/model_license_reviewer/output/model_license_review.csv
streamlit run run.py

Configuration

All configuration lives in app/config/settings.py and is overridable via environment variables:

• HF_EVAL_DATA_DIR — where sample/generated CSVs are written (default ./data)
• HF_EVAL_EXPORT_DIR — default export directory (default ./exports)
• HF_EVAL_BENCHMARK_CSV / HF_EVAL_LICENSE_CSV — real data source paths (optional)
• HF_EVAL_LOG_LEVEL — logging verbosity (default INFO)

Outputs

data/registry.csv, data/benchmark_results.csv, data/license_review.csv are generated on first run. Sample exports demonstrating the export feature are checked into exports/ (registry_full.csv, registry_full.json, recommendations.md).

Tests

pytest tests/ -v

17 tests covering: sample-data generation, registry schema, license-review classification rules (license normalization, commercial-use classification, privacy-risk detection, the approval engine), the recommendation engine (every recommendation has a written reason), and export correctness (valid CSV/JSON/Markdown).

Honest scope notes

The originating task description asked for a very large surface area (PDF/HTML export, 10+ chart types including heatmaps/box plots, a full multi-service microarchitecture, CI, etc.). This build prioritizes a fully working, tested, honestly-scoped implementation of the actual acceptance criteria in HF-01.07 (filtering, CSV/JSON export, benchmark charts, recommendation notes) plus a genuinely useful superset (comparison view, analytics, license dashboard, Markdown export) over a much larger pile of code that couldn't be verified to run in this environment. If you want any of the remaining items (PDF export, heatmap/box-plot charts, HTML report export, CI config), they slot cleanly into export_service.py / chart_service.py — say the word and I'll add them.

Future improvements

• PDF and HTML export (Markdown export is implemented; PDF/HTML would reuse report_generator patterns already in model_license_reviewer's notebook)
• Heatmap, box-plot, and trend charts in chart_service.py
• Live Hugging Face Hub search inside the dashboard (huggingface_hub.HfApi.list_models)
• CI (GitHub Actions) running pytest on push
• Authentication / multi-user support for an internal deployment