Neural Upgrade: Living Beyond Plan A