Repository: NVIDIA/NVFlare

Issue:
Error in aggregating models
I'm using NVFlare 2.0.6 and torchvision ResNet-50 model for training. The FL system (consisting 4 clients) had completed the training for the first round but when aggregating into the global model I faced this error. Please help me resolve this problem, thank you. 
```
2022-01-27 18:08:14,731 - InTimeAccumulateWeightedAggregator - INFO - [run=1, wf=scatter_gather_ctl]: aggregating 4 update(s) at round 0
Traceback (most recent call last):
  File "/home/jupyter-test/.conda/envs/fl/lib/python3.8/site-packages/nvflare/app_common/workflows/scatter_and_gather.py", line 202, in control_flow
    self._global_weights = self.shareable_gen.shareable_to_learnable(aggr_result, fl_ctx)
  File "/home/jupyter-test/.conda/envs/fl/lib/python3.8/site-packages/nvflare/app_common/shareablegenerators/full_model_shareable_generator.py", line 60, in shareable_to_learnable
    weights[v_name] += v_value
numpy.core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
2022-01-27 18:08:14,813 - ScatterAndGather - ERROR - [run=1, wf=scatter_gather_ctl]: Exception in ScatterAndGather control_flow: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
Traceback (most recent call last):
  File "/home/jupyter-test/.conda/envs/fl/lib/python3.8/site-packages/nvflare/app_common/workflows/scatter_and_gather.py", line 202, in control_flow
    self._global_weights = self.shareable_gen.shareable_to_learnable(aggr_result, fl_ctx)
  File "/home/jupyter-test/.conda/envs/fl/lib/python3.8/site-packages/nvflare/app_common/shareablegenerators/full_model_shareable_generator.py", line 60, in shareable_to_learnable
    weights[v_name] += v_value
numpy.core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
2022-01-27 18:08:14,813 - ServerRunner - ERROR - [run=1, wf=scatter_gather_ctl]: Aborting current RUN due to FATAL_SYSTEM_ERROR received: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
2022-01-27 18:08:14,813 - ServerRunner - INFO - [run=1, wf=scatter_gather_ctl]: asked to abort - triggered abort_signal to stop the RUN
2022-01-27 18:08:14,813 - ServerRunner - INFO - [run=1, wf=scatter_gather_ctl]: Workflow: scatter_gather_ctl finalizing ...
```


Discussion:
Modified the 60 line as follows in the **full_model_shareable_generator.py** did solve the problem, but I haven't closed this issues yet in order to wait for the confirmation from developer:
`weights[v_name] += v_value`
to
`weights[v_name] = weights[v_name].astype('float64') + v_value.astype('float64')`

Resolving patch:
diff --git a/nvflare/app_common/shareablegenerators/full_model_shareable_generator.py b/nvflare/app_common/shareablegenerators/full_model_shareable_generator.py
--- a/nvflare/app_common/shareablegenerators/full_model_shareable_generator.py
+++ b/nvflare/app_common/shareablegenerators/full_model_shareable_generator.py
@@ -57,7 +57,7 @@ def shareable_to_learnable(self, shareable: Shareable, fl_ctx: FLContext) -> Mod
             if dxo.data is not None:
                 model_diff = dxo.data
                 for v_name, v_value in model_diff.items():
-                    weights[v_name] += v_value
+                    weights[v_name] = weights[v_name] + v_value
         elif dxo.data_kind == DataKind.WEIGHTS:
             weights = dxo.data
             if not weights:


Test patch: