Yes, it is completely fine to charge the AirPods case with AirPods inside.