Repository: wandb/wandb

Issue:
[CLI]: UserWarning: The torch.cuda.*DtypeTensor constructors are no longer recommended
### Describe the bug

<!--- Description of the issue below  -->

Using wandb 0.15.9 on python 3.10.12 with torch 2.1.0.dev20230825+cu118, I get the following notice. (It's not an exception: the program continues running.)

<!--- A minimal code snippet between the quotes below  -->
```python
wandb.init(
    project=f'{PROJECT}',
    sync_tensorboard=True,
    notes=MODEL_PREFIX,
    config={
        'lr': LEARNING_RATE, 
        'model_version': net.VERSION,
    }
)
tbwriter = SummaryWriter(f'runs/{MODEL_PREFIX}') # Must come after wand init

#[training loop]
tbwriter.add_scalar('training_loss', training_loss, epoch)
```

<!--- A full traceback of the exception in the quotes below -->
```shell
/opt/conda/lib/python3.10/site-packages/wandb/wandb_torch.py:193: UserWarning: The torch.cuda.*DtypeTensor constructors are no longer recommended. It's best to use methods such as torch.tensor(data, dtype=*, device='cuda') to create tensors. (Triggered internally at ../torch/csrc/tensor/python_tensor.cpp:83.)
  check = torch.cuda.FloatTensor(1).fill_(0)
```


### Additional Files

_No response_

### Environment

WandB version: 
0.15.9 

OS:
Debian 11

Python version: 
3.10.12

Versions of relevant libraries: 
torch 2.1.0.dev20230825+cu118

### Additional Context

_No response_


Discussion:

Resolving patch:
diff --git a/wandb/wandb_torch.py b/wandb/wandb_torch.py
--- a/wandb/wandb_torch.py
+++ b/wandb/wandb_torch.py
@@ -147,69 +147,55 @@ def add_log_gradients_hook(
     def log_tensor_stats(self, tensor, name):
         """Add distribution statistics on a tensor's elements to the current History entry"""
         # TODO Handle the case of duplicate names.
-
-        if isinstance(tensor, tuple) or isinstance(tensor, list):
-            while (isinstance(tensor, tuple) or isinstance(tensor, list)) and (
-                isinstance(tensor[0], tuple) or isinstance(tensor[0], list)
+        if isinstance(tensor, (tuple, list)):
+            while isinstance(tensor, (tuple, list)) and isinstance(
+                tensor[0], (tuple, list)
             ):
                 tensor = [item for sublist in tensor for item in sublist]
-            tensor = torch.cat([t.reshape(-1) for t in tensor])
+            tensor = torch.cat([t.detach().clone().reshape(-1) for t in tensor])
 
+        tensor = tensor.detach().clone()
         # checking for inheritance from _TensorBase didn't work for some reason
         if not hasattr(tensor, "shape"):
             cls = type(tensor)
             raise TypeError(f"Expected Tensor, not {cls.__module__}.{cls.__name__}")
 
-        # HalfTensors on cpu do not support view(), upconvert to 32bit
-        if isinstance(tensor, torch.HalfTensor):
-            tensor = tensor.clone().type(torch.FloatTensor).detach()
-
         # Sparse tensors have a bunch of implicit zeros. In order to histo them correctly,
         # we have to count them up and add them to the histo ourselves.
         sparse_zeros = None
         if tensor.is_sparse:
             # Have to call this on a sparse tensor before most other ops.
-            tensor = tensor.cpu().coalesce().clone().detach()
+            tensor = tensor.cpu().coalesce()
 
             backing_values = tensor._values()
-            non_zero_values = backing_values.numel()
-            all_values = tensor.numel()
-            sparse_zeros = all_values - non_zero_values
+            sparse_zeros = tensor.numel() - backing_values.numel()
             tensor = backing_values
 
         flat = tensor.reshape(-1)
 
-        # For pytorch 0.3 we use unoptimized numpy histograms (detach is new in 0.4)
-        if not hasattr(flat, "detach"):
-            tensor = flat.cpu().clone().numpy()
-            wandb.run._log({name: wandb.Histogram(tensor)}, commit=False)
-            return
-
         if flat.is_cuda:
-            # TODO(jhr): see if pytorch will accept something upstream to check cuda support for ops
-            # until then, we are going to have to catch a specific exception to check for histc support.
             if self._is_cuda_histc_supported is None:
-                self._is_cuda_histc_supported = True
-                check = torch.cuda.FloatTensor(1).fill_(0)
                 try:
-                    check = flat.histc(bins=self._num_bins)
-                except RuntimeError as e:
-                    # Only work around missing support with specific exception
-                    # if str(e).startswith("_th_histc is not implemented"):
-                    #    self._is_cuda_histc_supported = False
-                    # On second thought, 0.4.1 doesnt have support and maybe there are other issues
-                    # lets disable more broadly for now
+                    flat.histc(bins=self._num_bins)
+                except RuntimeError:
                     self._is_cuda_histc_supported = False
-
-            if not self._is_cuda_histc_supported:
-                flat = flat.cpu().clone().detach()
+                else:
+                    self._is_cuda_histc_supported = True
 
             # As of torch 1.0.1.post2+nightly, float16 cuda summary ops are not supported (convert to float32)
-            if isinstance(flat, torch.cuda.HalfTensor):
-                flat = flat.clone().type(torch.cuda.FloatTensor).detach()
+            if not self._is_cuda_histc_supported:
+                flat = flat.cpu()
+            elif not isinstance(
+                flat, (torch.cuda.FloatTensor, torch.cuda.DoubleTensor)
+            ):
+                flat = flat.type(torch.cuda.FloatTensor)
 
-        if isinstance(flat, torch.HalfTensor):
-            flat = flat.clone().type(torch.FloatTensor).detach()
+        # Since we use histc, we need to make sure that torch supports the operation on CPU,
+        # otherwise we'll get a runtime error. Hence, we need to upcast to float32.
+        if not flat.is_cuda and not isinstance(
+            flat, (torch.FloatTensor, torch.DoubleTensor)
+        ):
+            flat = flat.type(torch.FloatTensor)
 
         # Skip logging if all values are nan or inf or the tensor is empty.
         if self._no_finite_values(flat):
@@ -229,7 +215,7 @@ def log_tensor_stats(self, tensor, name):
         if tmin > tmax:
             tmin, tmax = tmax, tmin
         tensor = flat.histc(bins=self._num_bins, min=tmin, max=tmax)
-        tensor = tensor.cpu().clone().detach()
+        tensor = tensor.cpu().detach().clone()
         bins = torch.linspace(tmin, tmax, steps=self._num_bins + 1)
 
         # Add back zeroes from a sparse tensor.


Test patch: