MinhNH232331M/master-thesis

Face verification research pipeline

Implements the research protocol and implementation plan. The project separates fitting, validation, operating-threshold calibration, and final testing. The provided train/test split JSON files are never read by the pipeline.

Environment

The verified local base is C:/Users/ADMIN/miniconda3/envs/thesis/python.exe, Python 3.13.11, PyTorch 2.9.1+cu126, torchvision 0.24.1+cu126, and a GTX 1650 Ti with 4 GB VRAM. The project-local .venv inherits the existing packages and adds pytorch-metric-learning==2.9.0; the base environment is unchanged.

From the repository root in PowerShell:

$PY = ".venv/Scripts/python.exe"
& $PY run_experiment.py doctor --config configs/thesis.yaml

For a new environment, install this project's dependencies with python -m pip install -e ".[test]" after installing a suitable PyTorch/torchvision build. On the current machine, the local environment is already prepared. To recreate an environment sharing the existing conda packages:

$BASE = "C:/Users/ADMIN/miniconda3/envs/thesis/python.exe"
& $BASE -m venv --without-pip --system-site-packages .venv
& $BASE -m pip install --no-deps --target .venv/Lib/site-packages pytorch-metric-learning==2.9.0

The backbone is vendored from InsightFace commit d9cb10e6bda109bc9b4a6ca8b92d7f3342b56492; its source hash and MIT license are under face_research/vendor. Only the adapter's forward method changes AMP ownership; the upstream state keys and graph are preserved. The ArcFace package owns the trainable class-weight matrix, which is included in the optimizer. Configuration margins use radians and are converted to the package's degrees.

Run a complete smoke test

& $PY run_experiment.py create-synthetic
& $PY run_experiment.py multirun --config configs/smoke.yaml --artifact-dir artifacts/smoke_v1

The default smoke run covers all three folds and all five conditions, with one training seed, three optimizer updates, ten bootstrap replicates, and a small randomly initialized model. Add --folds 0 for a shorter single-fold integration check. Synthetic images are random colored textures, not face photographs. Outputs are explicitly labeled smoke and written to smoke_exports; they are not evidence about recognition performance.

Repeat the same command to resume matching artifacts. Training and bootstrap checkpoints are reused, and cached embeddings are reused. Configuration/input mismatches require a fresh artifact directory.

Prepare the real data

& $PY run_experiment.py prepare --config configs/thesis.yaml --artifact-dir artifacts/data_audit_v1

Preparation reads only the complete gender metadata and processed image tree. It decodes images, checks exact RGB-pixel duplicates, rejects invalid entries, and retains identities with at least two unique usable images. Cross-person duplicate images are removed from every implicated person; the code does not infer that those people are the same person. Within-person exact duplicates are reduced to one deterministic representative.

prepare/audit.json records every exclusion, unmatched folder, source/gender allocation, and split fallback. Exact duplicate checks do not detect all repeated people or near-duplicate frames: review the source records before setting identity_audit_verified. No identity aliases or gender corrections are guessed.

Optional data.corrections points to a JSON object keyed by an original metadata id. Supported fields are path (relative to data.root), gender, person_id (a verified canonical person ID), and exclude (boolean). Unknown IDs/fields and conflicting canonical gender labels are rejected. Example schema:

{
  "images/collection/original_folder": {
    "path": "images/collection/corrected_folder",
    "gender": "female",
    "person_id": "verified_person_001"
  }
}

Use corrections only when justified by authoritative records. Images and input metadata are never modified. Eligibility changes require a new artifact directory.

The first real-data preparation found 28 exact RGB duplicate groups spanning 11 folder pairs. After canonicalizing those reviewed aliases, conservatively excluding 11 additional plausible alias pairs, and resolving the invalid metadata record by explicit exclusion, the final preparation retained 919 identities (711 male, 208 female) and 25,491 images. These are audit counts, not model results. See the identity audit.

Preparation creates:

• Identity and image inventories with stable IDs and content hashes.
• Three outer test folds (seed 42), calibration splits (1000 + fold), and validation splits (2000 + fold).
• Nested, collection-aware condition subsets for run seeds 42, 43, and 44.
• Fixed genuine and MM/FF/MF impostor comparison manifests for validation, calibration, and testing.

The script uses joint gender/collection stratification where feasible and records a gender-only fallback otherwise. An absent gender or infeasible controlled condition is an error. Pair manifests cover every distinct eligible identity pair and sample up to the configured number of image comparisons per identity pair. Genuine comparisons sample different images of the same identity. No reliable session annotations are currently supplied; the implemented pair policy is within available captures, not guaranteed cross-session verification.

Pretrained checkpoint and scientific configuration

configs/thesis.yaml contains development starting points, not selected thesis hyperparameters. Before final evaluation:

1. Supply a compatible pretrained iResNet50 PyTorch state dictionary in model.checkpoint, its SHA-256 in model.sha256, and its provenance in model.source.
2. Verify the person audit and processed-image provenance. Set data.identity_audit_verified and data.preprocessing_verified only when those checks are complete.
3. Complete fold-local development, record the chosen settings, and set frozen: true. Generate the final study artifacts in a fresh directory.

The study checkpoint is now present at models/pretrained_ckpt/backbone_r50.pth, with SHA-256 ef005ba502a2d0f40e7b94245054d3a23e375776f7035080fcbce45ae49cfb01. It strictly loads all 475 state entries into the vendored iResNet50. Its private mirror, source association, and preprocessing limits are recorded in the provenance note. The implementation does not silently replace it with random weights, an ONNX inference model, or a different ResNet architecture.

For development-only experiments, copy a configuration with smoke: true, architecture: iresnet50, and the actual pretrained checkpoint; all resulting reports remain labeled smoke. Use only that fold's fitting/validation results for decisions. Final scientific calibration/testing requires the frozen configuration.

The input adapter uses Pillow RGB conversion, EXIF orientation, bilinear antialiased resize to 112 × 112, and (pixel / 255 - 0.5) / 0.5. It consumes the existing processed images and does not perform face detection or alignment. Confirm those images and this resize policy match the checkpoint before final evaluation. There is no silent detector fallback.

Local precision: FP32 forward/backward is the intended local configuration. FP16 produced non-finite activations on the tested GTX 1650 Ti/software combination, so amp: false is the default. AMP remains opt-in for hardware on which preflight passes. Small microbatches and gradient accumulation are supported; accumulation does not emulate large-batch BatchNorm. Frozen running statistics are the adaptation development default, with trainable affine parameters where upstream allows them.

Individual stages and full study

& $PY run_experiment.py train --config configs/thesis.yaml --artifact-dir artifacts/study_v1 --fold 0 --condition M50 --seed 42
& $PY run_experiment.py extract --config configs/thesis.yaml --artifact-dir artifacts/study_v1 --fold 0 --condition M50 --seed 42 --role select
& $PY run_experiment.py validate --config configs/thesis.yaml --artifact-dir artifacts/study_v1 --fold 0 --condition M50 --seed 42
& $PY run_experiment.py extract --config configs/thesis.yaml --artifact-dir artifacts/study_v1 --fold 0 --condition M50 --seed 42 --role operate
& $PY run_experiment.py calibrate --config configs/thesis.yaml --artifact-dir artifacts/study_v1 --fold 0 --condition M50 --seed 42
& $PY run_experiment.py extract --config configs/thesis.yaml --artifact-dir artifacts/study_v1 --fold 0 --condition M50 --seed 42 --role test
& $PY run_experiment.py evaluate --config configs/thesis.yaml --artifact-dir artifacts/study_v1 --fold 0 --condition M50 --seed 42

Run prepare with the same configuration/directory before individual stages. Validation reports verification AUROC on unseen identities; it never evaluates unseen people against the local training classification head. It is a development diagnostic, not a final metric or an automatic hyperparameter search.

The normal complete study command is:

& $PY run_experiment.py multirun --config configs/thesis.yaml --artifact-dir artifacts/study_v1

For the frozen thesis study, tools/run_frozen_study.py additionally verifies and snapshots the development selection and checkpoint preflight, resumes the multirun in artifacts/thesis_study_v1, and synthesizes references/p2_face_dataset_experiment_report.md after all 39 evaluations and 2,000 bootstrap replicates complete.

This schedules 36 adaptation fits and three untouched-baseline evaluations. M0 is evaluated once per outer fold and is paired to each corresponding adapted run for contrasts; it is not treated as having independent training seeds.

Use train --resume to resume individual training. --stop-after N pauses at N successful updates without changing the configured final budget. Checkpoints include head/backbone, optimizer, schedule, AMP scaler, random states, sample position and hashes. Mismatched settings, pretrained weights, or manifests are rejected. Non-finite loss/gradients fail explicitly. A run is marked complete only after all configured successful updates.

The complete runner saves failed/running/complete states. It never performs parameter selection from test results. It supports a single CPU or CUDA device; distributed training and automatic hyperparameter search are outside this implementation.

Evaluation and uncertainty

The implementation applies the protocol's exact finite-population category weights to the same comparisons. It first normalizes within genuine identities or impostor identity pairs, then within demographic categories. Reweighting changes global rates while preserving subgroup rates at a fixed threshold.

Thresholds use score >= threshold; tied impostor scores are excluded together. A threshold immediately above an excluded tie group is represented using float64 nextafter. Score CSVs round-trip float64 precision. Reports separate the calibration target, achieved calibration FPR, and achieved test FPR.

Every model produces eight primary rows: two FPR targets × two calibration populations × two test populations. Additional files contain low-FPR tail support and balanced-subsampling sensitivity results.

The bootstrap resamples people within gender/collection strata. One multiplicity per person is shared across every model, seed and fold role. Genuine contributions use that multiplicity once; impostor contributions use the product of both endpoints. Thresholds are recalibrated in each replicate. Undefined categories yield invalid estimates, whose counts are reported; they are never replaced with zero. Intervals with invalid replicates use the remaining defined replicates and must be interpreted alongside their validity counts. Intervals condition on the fitted models and pair manifests, rather than claiming to measure retraining uncertainty.

summarize aggregates paired contrasts within folds/seeds before averaging. Its pooled standard deviation is descriptive run variation, and separate exports report training-seed spread within folds and spread across fold means. Bootstrap replicates are cached separately and resumable, enforce a 95% minimum valid-replicate fraction, and preserve missing-support reasons. Score tables use compact categorical endpoint storage so all 39 paired evaluations fit in memory.

Scientific exports are under thesis_exports; incomplete fold/run selections go to partial_exports; every smoke configuration uses smoke_exports. Main files include:

• per_run_metrics.csv, main_results_mean_std.csv, and bootstrap_intervals.csv.
• sensitivity_per_run.csv and sensitivity_mean_std.csv.
• tail_support_per_run.csv, tail_support_summary.csv, and test_support_per_run.csv.
• Subgroup and population-comparison figures in PDF and PNG.
• summary.json with completeness, uncertainty interpretation, and invalid-bootstrap counts.

Verification

& $PY -m pytest -q

Tests cover fixed splits and leakage, exclusion of supplied split files, nested condition counts, duplicate handling, hand-calculated weighted rates, tied thresholds, independent calibration/test scores, bootstrap endpoints, optimizer updates, exact CPU resume, strict checkpoint loading, the complete synthetic workflow, and a CUDA iResNet50 forward/backward pass when a GPU is present. The GPU test uses random weights and training-mode BatchNorm; frozen-statistics adaptation must also be checked with the chosen pretrained checkpoint.

Full thesis training has not been run. Code verification and synthetic outputs must not be interpreted as thesis findings.

Implementation verification on 2026-09-06: all 20 tests passed. The standalone three-fold smoke run completed 15 evaluations (five conditions per fold), saved ten bootstrap replicates, and exported five figures in each of PDF and PNG. The exact pretrained checkpoint passed an FP32 CUDA preflight at microbatch 16 with finite [16, 512] embeddings, finite loss, and nonzero backbone/head gradients under frozen batch-normalization statistics. Fold-local development using only fit and select chose 2,000 updates in all three folds. The final scientific-prerequisite check reports configuration_ready: true.