Cleanup: remove deprecated fields the proper way

Also move converters to their own file to make it easy to move them into v1beta1 later.
I knew you would love that one. OK, real fix incoming.