Upgrade fixes


WIP

Addresses:

* https://tickets.corp.opscode.com/browse/OC-11572