[1.5] Update automation script to also handle the new application-based structure