# Known Issues — Cleanup Pass Required

This file tracks items flagged during the file-move pass. None of these block running
the code in its current form, but all require attention before release.

---

## ~~[ANNOTATORS] Undocumented conda env requirements~~ — RESOLVED

All annotators now documented in tiered dependency model. See `annotators/README.md`.
Base annotators (wdv3_tagger, gradcam, lama_inpainter) install with `pip install -e annotators/`.
Optional extras: `[bizarre_tagger]`, `[lang_sam]`, `[animeinsseg]`.

---

## [SCRIPTS] Ambiguous ownership

- `scripts/benchmark.py` — assigned to `training/scripts/`; may also apply to inference outputs
- `scripts/demo_iter1.py` — assigned to `inference/scripts/` (imports semanticsam + layerdiffuse + live2d)
- `scripts/vis_gradcam.py` — assigned to `inference/scripts/`; depends on `annotators/gradcam.py` (separate plugin)
- `scripts/vis_compare_partextr.py` — assigned to `inference/scripts/`; touches annotator outputs

---

## ~~[MODULES] Import graph not verified~~ — RESOLVED

Verified 2026-03-25: all 59 common/ modules import cleanly (v2 verification, Phase 1: 59/59 PASS).
No circular dependencies found.

---

## [HARD-CODED PATHS]

- `animation_demo/animation_demo.py` lines 26, 28: hard-coded `"configs/ad/mvc_base_cfg.yaml"`
  and `"configs/ad/examples/config/mvc/interactive_window_example.yaml"` — after the move,
  the `ad/` prefix is gone; paths must be updated to `configs/mvc_base_cfg.yaml` and
  `configs/examples/...`
- `animation_demo/animation_demo.py` lines 39, 40, 46: hard-coded `workspace/datasets/...` paths
  — **mitigated** by running from the repo root where `workspace/` is a direct child
  (line 46 sets `src_dir` and is also a hard-coded workspace path)
- Root `README.md` line 17: hard-coded `/home/ljsabc/live2d_parsing/` path (original repo only)

---

## ~~[UI] requirements.txt version conflict~~ — RESOLVED

Resolved by unified env (2026-03-24). All sub-codebases now share a single
`requirements.txt` at repo root with Python 3.12. Legacy per-subdir requirements
renamed to `requirements-legacy-3.10.txt`.

---

## [COMMON] utils/ split candidates (deferred)

- `utils/sampler.py` — likely training-only; review in cleanup pass
- `utils/ssim_torch.py` — likely training-only
- `utils/torchcv.py` — **do NOT move without verification**: `inference/scripts/vis_texture.py`
  imports `from utils.torchcv import cluster_inpaint_part`; moving this would break inference

---

## [ANNOTATORS] mmdet3.yml hardcoded developer path

- `annotators/animeinsseg/mmdet3.yml` line 134: `prefix: /home/jlin/miniconda3/envs/mmdet3`
  — hardcoded developer path. The `mmdet3` conda env is now redundant (merged into
  `live2d_inference`), so this YAML is a historical artifact. Low priority — only affects
  users who try to recreate the env via conda YAML instead of pip.

---

## ~~[ANNOTATORS] Root annotators/README.md env name inconsistency~~ — RESOLVED

README rewritten around tiered dependency model. Env names: `see_through_dev` (main),
`ann_mmpose` (legacy face detection).

---

## [INFERENCE] `talking_head` stale imports

- `inference/scripts/extr_psd.py` line 45: `from talking_head.preprocess import further_extr`
- `inference/scripts/extr_biseg.py` line 16: `from talking_head.preprocess import *`
- `inference/scripts/pick_cmp_rows.py`: imports `talking_head.preprocess` at module level
- `inference/scripts/vis_segs.py`: imports from `talking_head` (cross-env)
- `inference/scripts/vis_texture.py`: imports from `talking_head` (cross-env)
- `talking_head` lives in `animation_demo/` (separate `live2d_animation` env, Python 3.10)
- The `further_extr` function also exists at `common/utils/inference_utils.py:393`
- These scripts also depend on `smug` (external face-parsing package, not in requirements)
- `extr_biseg.py` fails at import time in the unified `see_through_dev` env
- `extr_psd.py` imports succeed at module level (`--help` works); `talking_head` import
  is inside a function — fails at runtime only, not at import time
- Confirmed 2026-03-25 (v2 verification): all 5 scripts above fail in `see_through_dev` env

---

## ~~[INFERENCE] `vis_gradcam.py` stale imports~~ — RESOLVED

- Fixed 2026-03-25 (v3 verification): added `training/scripts` to sys.path,
  changed import to `from data_pipeline import ...`
- `live2d.scrap_model` resolves via `common/` editable install (was never broken)

---

## [INFERENCE] `smug` undocumented dependency

- `inference/scripts/extr_psd.py` and `inference/scripts/further_extr.py` import `smug`
  (face landmark detection + IRIAMS separation)
- `smug` is not in requirements.txt, not in the repo, and not on PyPI
- Appears to be a private/internal package — needs a tier requirement file or documentation

---

## [TRAINING] `train_layerdiff.py` gradient_checkpointing bug

- Line 707: `raise Exception('Not implemented')` blocks gradient checkpointing
- Working implementation exists at lines 873-874
- Prevents single-GPU training of SDXL UNet (requires 4-GPU DeepSpeed ZeRO-2)
- Confirmed 2026-03-24: OOM on 48GB GPU at target_size=256, batch_size=1

---

## [TRAINING] `train_layerdiff3d.py` missing checkpoint prerequisite

- `--pretrained_unet` defaults to None — requires a pre-converted 3D UNet checkpoint
- Must run `scripts/unet2d_to_3d.py` first to produce the checkpoint (with `layerdiff3d.json` config)
- Not documented in README or training guide
- Verified 2026-03-25: full chain works when using correct SDXL config (layerdiff3d.json)

---

## ~~[UI] Missing `requirements_ui.txt` crashes launch~~ — RESOLVED

Removed vestigial `prepare_environment()` from `ui/ui/launch.py`. The embedded
pip-install mechanism was from the old monolithic repo; all deps are now handled
by the unified `requirements.txt` at repo root.

---

## [TRAINING] pip `training` namespace collision — NOT REPRODUCED in v3

- v2 verification found an orphaned `training` package (Meta Platforms, Inc.)
  at `site-packages/training/` which shadowed the local `training/` directory
- v3 verification (2026-03-25): NOT REPRODUCED across 4 tier installs
  (base, deepspeed, annotators, mmdet). Checked after each tier — never appeared.
- May have been caused by a transient dependency or different install order in v2
- Keep monitoring but no permanent fix needed at this time

---

## [INFERENCE] Module-level side effects in scripts

- `inference/scripts/demo_iter1.py`: loads semanticsam model at import time (before argparse)
- `inference/scripts/vis_compare_partextr.py`: calls `os.listdir()` on hardcoded dataset path at import time
- ~~`inference/scripts/vis_samask.py`: calls `os.makedirs()` at import time~~ — fixed (exist_ok=True)
- The remaining two scripts fail `--help` because side effects run before argument parsing
- Low priority — these are utility/visualization scripts, not core pipeline

---

## [TRAINING] `unet2d_to_3d.py` module-level model loading

- `training/scripts/unet2d_to_3d.py` loads a model checkpoint at module level
- `--help` fails if `workspace/training_output/finetune_marigolddepth_iter1` doesn't exist
- Related to the existing `train_layerdiff3d.py` prerequisite issue above

---

## [DEPS] Unresolved missing dependencies

The following packages are imported but not available in any requirements file.
These are deferred for manual review — do not add without understanding the
licensing, size, and maintenance implications.

- **mmpose**: Required by `annotators/anime_face_detector/detector.py` (eager import in `__init__.py`).
  Face detection annotator is non-functional without it. Consider lazy import guard or separate tier file.
- **sam3**: Required by `inference/scripts/sam_inf.py` and `inference/scripts/sam3_inf.py`.
  Package not on PyPI — may be private or renamed. Scripts are non-functional without it.
- **torchmetrics**: Required by `training/scripts/benchmark.py` for FID evaluation.
  Optional — only needed for evaluation, not training.

---

## ~~[TRAINING] `unet2d_to_3d.py` → `train_layerdiff3d.py` architecture mismatch~~ — RESOLVED (testing error)

- V3 verification incorrectly chained marigold (SD1.5) → unet2d_to_3d (marigold3d.json) → train_layerdiff3d (SDXL)
- The correct chain is: train_layerdiff (SDXL) → unet2d_to_3d (layerdiff3d.json) → train_layerdiff3d (SDXL)
- Two separate pipelines exist, each with its own config:
  - Marigold depth: `marigold.json` (SD1.5, 1024) → `marigold3d.json` (SD1.5 3D, 1024)
  - LayerDiff: SDXL 2D UNet → `layerdiff3d.json` (SDXL 3D, 2048)
- Re-verified 2026-03-25: full SDXL chain PASS (conversion tensor diff=0, 100 steps stable)
- Remaining issue: `unet2d_to_3d.py` has no argparse — all paths/config hardcoded for marigold pipeline only

---

## [TRAINING] `unet2d_to_3d.py` stale method name

- ~~Line 75: `model.init_temporal_weights()` — method does not exist on `UNetFrameConditionModel`~~
- Fixed 2026-03-25 (v3 verification): changed to `model.init_weights()`

---

## [TRAINING] `kepler.py` module-level test code

- `training/train/kepler.py` contains module-level test code (line 232+)
- Fails with `ValueError: not enough values to unpack (expected 3, got 2)` from quantizer
- Import as module works fine — only fails when executed as `__main__`

---

## [INFERENCE] `infer_instanceseg.bash` data processing issue

- `parse_live2d.py instance_segmentation` fails with empty filename during exec_list processing
- Likely a format mismatch between exec_list entries and expected input format
- Other subcommands (infer_bizarre_tagger, parse_live2d, infer_langsam) work with the same exec_list