I ram this script:
find -type f|while read a; do cat "$a"|grep oid|cut -d" " -f2|sponge "$a"; done
jdupes . -r
which toldnme that these files are identical.
./kl-f8-anime2.vae.pt
./vaedelicatecolcors_v10.pt
./babes_11.vae.pt
./vae-ft-mse-840000-ema-pruned.ckpt
./Anything-V3.0.vae.pt
./Counterfeit-V2.5.vae.pt
./NovelAI-vae via hf.co_Reviem/diffusion_pytorch_model.bin
If you're storing on cloud storage, you can deduplicate with shortcuts (google drive) or notes; on linux you can use hard links (rm vaedelicatecolors_v10.pt && ln kl-f8-anime2.vae.pt vaedelicatecolors_v10.pt. if youre storing in a git repo, you can just use a sparse checkout like I do (the lfs backend will dedupe serverside because thats how it works, at least on HF)