Repository: pytorch
Issue: Parameters between models don't copy in the C++ Pytroch Frontend under windows
Labels: module: windows,module: cpp,triaged

### 🐛 Describe the bug

Like I mentioned gradient don't copy either correctly (and there are not much examples to work with how to do this )
When I do it like in this example https://github.com/pytorch/examples/blob/main/cpp/distributed/dist-mnist.cpp line 127 (which just simple manipulate for distributed models)

What I do is a DNQ net where I partial transfer the gradient between target and policy  model but I done a project before so I know it doesn't work then either.

This my last code for it I done a lot of iterations to see how to solve it.


```cpp
auto target_params_n = target_model->named_parameters(true);
auto policy_params_n = policy_model->named_parameters(true);

for(auto& policy_param : policy_params_n)
{
	for(auto& target_param : target_params_n)
	{
		if(target_param.key() == policy_param.key())
		{
			if(target_param.value().grad().defined() && policy_param.value().grad().defined())
			{
				auto& t_grad_param = target_param.value().grad(); 

				if(t_grad_param.defined())
				{
					auto updated_grad = policy_param.value().grad().data() * ( 1.0 - TAU ) + target_param.value().grad().data() * TAU;
					// Update the target network parameters' gradients slowly towards the policy network parameters
					target_param.value().grad().data() = updated_grad.clone().detach();
				}
			}
		}
	}
}	
```



This is since at least Pytorch 2.0 this 2.1.1 I hoped you solved so no.

I also tracked the change :
 `
```cpp
if(env.isReplayBufferReady())
{
	env.sampleReplayBuffer(policy_model , GAMMA);
	if(once_init == false)
	{
		for(const auto& named_param : policy_model->named_parameters(true))
		{
			const std::string name = named_param.key();
			const torch::Tensor param = named_param.value();
			if(param.requires_grad())
			{
				initialParams_policy[name] = param.clone().detach();
			}
		}
		once_init = true;
		is_init = true;
	}
}
//std::cout << "Reward: " << reward << std::endl;
prev_reward = prev_reward + reward;
reward = 0;
auto target_params_n = target_model->named_parameters(true);
auto policy_params_n = policy_model->named_parameters(true);

for(auto& policy_param : policy_params_n)
{
	for(auto& target_param : target_params_n)
	{
		if(target_param.key() == policy_param.key())
		{
			if(target_param.value().grad().defined() && policy_param.value().grad().defined())
			{
				auto& t_grad_param = target_param.value().grad(); 

				if(t_grad_param.defined())
				{
					auto updated_grad = policy_param.value().grad().data() * ( 1.0 - TAU ) + target_param.value().grad().data() * TAU;
					// Update the target network parameters' gradients slowly towards the policy network parameters
					target_param.value().grad().data() = updated_grad.clone().detach();
				}
			}
		}
	}
}	

if(is_init)
{
	for(const auto& named_param : policy_model->named_parameters(true))
	{
		const std::string name = named_param.key();
		const torch::Tensor param = named_param.value();
		if(param.requires_grad())
		{
			const torch::Tensor initial_param = initialParams_policy[name];
			const torch::Tensor change = torch::norm(param - initial_param);
			const torch::Tensor norm_initial_param = torch::norm(initial_param);
			const float change_percentage = ( change / norm_initial_param ).item<float>() * 100.0;
			total_percentage_change += change_percentage;
			param_count += 1;
			//	std::cout << name << ": " << std::fixed << std::setprecision(4) << change_percentage << "%" << std::endl;
		}
	}
	auto average_percentage_change = total_percentage_change / ( float )param_count;
	std::cout << "Average Percentage Change Policy: " << std::fixed << std::setprecision(4) << average_percentage_change << "%" << std::endl;
	param_count = 0;
	total_percentage_change = 0.0;
}
for(const auto& named_param : target_model->named_parameters(true))
{
	const std::string name = named_param.key();
	const torch::Tensor param = named_param.value();
	if(param.requires_grad())
	{
		const torch::Tensor initial_param = initialParams_target[name];
		const torch::Tensor change = torch::norm(param - initial_param);
		const torch::Tensor norm_initial_param = torch::norm(initial_param);
		const float change_percentage = ( change / norm_initial_param ).item<float>() * 100.0;
		total_percentage_change += change_percentage;
		param_count += 1;
		//	std::cout << name << ": " << std::fixed << std::setprecision(4) << change_percentage << "%" << std::endl;
	}
}
auto average_percentage_change = total_percentage_change / ( float )param_count;
std::cout << "Average Percentage Change Target: " << std::fixed << std::setprecision(4) << average_percentage_change << "%" << std::endl;
param_count = 0;
total_percentage_change = 0.0;
```

For test I set TAU to 0.5 . While I send the policy model into training the result were : 
.
.
.
sampleReplayBuffer
Average Percentage Change Policy: 2.0661%
Average Percentage Change Target: 0.0000%
In Step From: 48 To: 40
rnbqkb1r/1pp1pppp/p4n2/1N1p4/8/P4P2/1PPPP1PP/R1BQKBNR b KQkq - 0 5 moves a2a3
bestmove: a6b5
sampleReplayBuffer
Average Percentage Change Policy: 2.2029%
Average Percentage Change Target: 0.0000%
In Step From: 49 To: 33
rnbqkb1r/1pp1pppp/5n2/1p1p4/1P6/P4P2/2PPP1PP/R1BQKBNR b KQkq b3 0 6 moves b2b4
bestmove: e7e5
sampleReplayBuffer
Average Percentage Change Policy: 2.3678%
Average Percentage Change Target: 0.0000%
In Step From: 54 To: 46
rnbqkb1r/1pp2ppp/5n2/1p1pp3/1P6/P4PP1/2PPP2P/R1BQKBNR b KQkq - 0 7 moves g2g3
bestmove: c8d7
sampleReplayBuffer
Average Percentage Change Policy: 2.6246%
Average Percentage Change Target: 0.0000%
In Step From: 45 To: 37
rn1qkb1r/1ppb1ppp/5n2/1p1pp3/1P3P2/P5P1/2PPP2P/R1BQKBNR b KQkq - 0 8 moves f3f4
bestmove: e5f4
sampleReplayBuffer 

This is just the end nothing gets copied while I made sure the lines get executed. The fatal thing is there is no error message that something went wrong. I mean when this is not windows specific this could give rise to dozens error reports which more like knock on effects. Usually an automated test should have caught this.

### Versions

PyTorch version: 2.1.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Home
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.27.7
Libc version: N/A

Python version: 3.11.6 (tags/v3.11.6:8b6ee5b, Oct  2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: True
CUDA runtime version: 12.1.66
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080
Nvidia driver version: 546.17
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=3801
DeviceID=CPU0
Family=107
L2CacheSize=4096
L2CacheSpeed=
Manufacturer=AuthenticAMD
MaxClockSpeed=3801
Name=AMD Ryzen 7 5800X 8-Core Processor
ProcessorType=3
Revision=8448

Versions of relevant libraries:
[pip3] numpy==1.26.1
[pip3] torch==2.1.0+cu121
[pip3] torchaudio==2.1.0+cu121
[pip3] torchvision==0.16.0+cu121
[conda] Could not collect



cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @iremyux @Blackhex @cristianPanaite @jbschlosser