Now actually working, but I understand that the combination of feature scaling for GD and GDT essentially makes the deformation cost zero. Seems it improves things significantly. Likely, the initialization procedure is good enough, and matching the thing everywhere works very well - especially with this level of deformation. So next step, removing the GDT entirely, introducing feature pyramids for scale, and if time is good enough, rotations.