Repository: pytorch
Issue: [FSDP2] _sharded_param_data is sitll on meta while sharded_param moved to cuda after calling initialize_parameters()
Labels: triaged,module: fsdp

### 🚀 The feature, motivation and pitch

_sharded_param_data is sitll on meta while sharded_param moved to cuda after calling initialize_parameters()

the workaround is model = model.to("cuda"). but want a better solution like adding `_sharded_param_data.to(device)` in `lazy_init`

repro
```
model = nn.Sequential(
    nn.Sequential(
        nn.Linear(10, 10, bias=False, device="cuda"),
        nn.Linear(10, 10, bias=False, device="meta"),
    ),
    nn.Linear(10, 10, bias=False, device="cuda"),
)
for m in model:
    fully_shard(m)
fully_shard(model)
# _sharded_param_data is still on meta device
model[0][1].to_empty(device="cuda")
nn.init.normal_(model[0][1].weight)
# calling model.to will move _sharded_param_data to cuda
# model = model.to("cuda")
inp = torch.randn((10, 10), device="cuda")
model(inp)
```
error
```
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/nn/modules/container.py", line 217, in forward
[rank0]:     input = module(input)
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/nn/modules/module.py", line 1561, in _call_impl
[rank0]:     args_kwargs_result = hook(self, args, kwargs)  # type: ignore[misc]
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/distributed/_composable/fsdp/_fsdp_state.py", line 46, in fsdp_hook_wrapper
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/distributed/_composable/fsdp/_fsdp_state.py", line 173, in _pre_forward
[rank0]:     args, kwargs = self._fsdp_param_group.pre_forward(module, args, kwargs)
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/distributed/_composable/fsdp/_fsdp_param_group.py", line 256, in pre_forward
[rank0]:     self.unshard()
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/distributed/_composable/fsdp/_fsdp_param_group.py", line 197, in unshard
[rank0]:     self._all_gather_result = foreach_all_gather(
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/utils/_contextlib.py", line 115, in decorate_context
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/distributed/_composable/fsdp/_fsdp_collectives.py", line 61, in foreach_all_gather
[rank0]:     torch._foreach_copy_(foreach_copy_dsts, all_gather_inputs)
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/_meta_registrations.py", line 3285, in _meta_foreach_inplace
[rank0]:     _meta_foreach_out_of_place(*args, _scalar_op=_scalar_op, **kwargs)
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/_meta_registrations.py", line 3237, in _meta_foreach_out_of_place
[rank0]:     result.append(_scalar_op(*each_args, *args[nlists:], **kwargs))
[rank0]:   File "/data/users/weif/pytorch-official/pytorch/torch/_ops.py", line 1024, in __call__
[rank0]:     return self_._op(*args, **(kwargs or {}))
[rank0]: NotImplementedError: Cannot copy out of meta tensor; no data!
```
### Alternatives

_No response_

### Additional context

_No response_

cc @zhaojuanmao @mrshenli @rohan-varma @awgu @fegin @penguinwu @kwen2501 @chauhang