Keep the size variants at the same Unicode location and only move the horizontal/vertical components to the BMP PUA (Davide's suggestion).