Unexplained pytorch datatype

username: joepareti54
cooked: <p>in this program, I start with a numpy array with default initialization. The array is then used within the Pytorch Dataset class, and then the dataset is used in Dataloader. Without specifying anything, the data turns out to be  <em>tensor, dtype=torch.float64</em>. Why?</p>
<pre><code class="lang-auto">import numpy as np
import torch
import torch.nn.functional as F
from torch import nn, optim
from torch.utils.data import Dataset
from torch.utils.data import DataLoader
from torchvision import transforms
np.random.seed(seed=115)
X = np.random.rand(10,3)
class Wine(Dataset):
    def __init__(self,X):
        self.x = X[:, 1:]
        self.y = X[:, 0]
        self.num_samples = X.shape[0]

    def __len__(self):
        return(self.num_samples)

    def __getitem__(self, index):
        return (self.x[index] , self.y[index] )
#        
WineDS = Wine(X)
#
print('size of train ds ',len(WineDS))
train_dataloader = DataLoader(WineDS, batch_size=2, shuffle=False)
#
train_features , train_labels = next(iter(train_dataloader))
print(train_features)
print('----')
print(train_labels)

</code></pre>

username: InnovArul
cooked: <p>It is because the default collation function is used in Dataloader if you do not provide one (using <code>collate_fn</code> option in the <code>DataLoader</code>). You can provide your own function to <code>collate_fn</code> if you want to handle the batch of data in some other way.</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/pytorch/blob/e36a8c1f137af6de7fc5f8d2160f0bb355738824/torch/utils/data/_utils/collate.py#L84-L130">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/e36a8c1f137af6de7fc5f8d2160f0bb355738824/torch/utils/data/_utils/collate.py#L84-L130" target="_blank" rel="noopener">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/e36a8c1f137af6de7fc5f8d2160f0bb355738824/torch/utils/data/_utils/collate.py#L84-L130" target="_blank" rel="noopener">pytorch/pytorch/blob/e36a8c1f137af6de7fc5f8d2160f0bb355738824/torch/utils/data/_utils/collate.py#L84-L130</a></h4>



    <pre class="onebox">      <code class="lang-py">
        <ol class="start lines" start="84" style="counter-reset: li-counter 83 ;">
            <li>def default_collate(batch):</li>
            <li>    r"""</li>
            <li>        Function that takes in a batch of data and puts the elements within the batch</li>
            <li>        into a tensor with an additional outer dimension - batch size. The exact output type can be</li>
            <li>        a :class:`torch.Tensor`, a `Sequence` of :class:`torch.Tensor`, a</li>
            <li>        Collection of :class:`torch.Tensor`, or left unchanged, depending on the input type.</li>
            <li>        This is used as the default function for collation when</li>
            <li>        `batch_size` or `batch_sampler` is defined in :class:`~torch.utils.data.DataLoader`.</li>
            <li>
            </li>
<li>        Here is the general input type (based on the type of the element within the batch) to output type mapping:</li>
            <li>
            </li>
<li>            * :class:`torch.Tensor` -&gt; :class:`torch.Tensor` (with an added outer dimension batch size)</li>
            <li>            * NumPy Arrays -&gt; :class:`torch.Tensor`</li>
            <li>            * `float` -&gt; :class:`torch.Tensor`</li>
            <li>            * `int` -&gt; :class:`torch.Tensor`</li>
            <li>            * `str` -&gt; `str` (unchanged)</li>
            <li>            * `bytes` -&gt; `bytes` (unchanged)</li>
            <li>            * `Mapping[K, V_i]` -&gt; `Mapping[K, default_collate([V_1, V_2, ...])]`</li>
            <li>            * `NamedTuple[V1_i, V2_i, ...]` -&gt; `NamedTuple[default_collate([V1_1, V1_2, ...]),</li>
            <li>              default_collate([V2_1, V2_2, ...]), ...]`</li>
        </ol>
      </code>
    </pre>


  This file has been truncated. <a href="https://github.com/pytorch/pytorch/blob/e36a8c1f137af6de7fc5f8d2160f0bb355738824/torch/utils/data/_utils/collate.py#L84-L130" target="_blank" rel="noopener">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


username: joepareti54
cooked: <p>thank you; how would you change the default so that the dataloader returns 32 bit data instead of 64?</p>

username: InnovArul
cooked: <aside class="quote no-group" data-username="joepareti54" data-post="1" data-topic="151532">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/b2d939/48.png" class="avatar"> joepareti54:</div>
<blockquote>
<p><code>train_features , train_labels = next(iter(train_dataloader))</code></p>
</blockquote>
</aside>
<p>A straightforward way is to call <code>.float()</code> on <code>train_features</code> and <code>train_labels</code>.</p>
<pre><code class="lang-auto">train_features = train_features.float()
train_labels = train_labels.float()
</code></pre>
<p>If you want to do this at <code>collation</code> level, you can write a custom collate function and pass it to <code>DataLoader</code> as below.</p>
<pre><code class="lang-auto">def collate_fn(batch):
    X, y = [], []
    for data, label in batch:
        X.append(torch.tensor(data))
        y.append(torch.tensor(label))
    
    return torch.stack(X).float(), torch.stack(y).float()

...
train_dataloader = DataLoader(WineDS, batch_size=2, shuffle=False, collate_fn=collate_fn)
</code></pre>

username: diegoaichele
cooked: <p>Something like this?</p>
<pre><code class="lang-auto">import numpy as np
import torch
import torch.nn.functional as F
from torch import nn, optim
from torch.utils.data import Dataset
from torch.utils.data import DataLoader
from torchvision import transforms
np.random.seed(seed=115)
X = np.random.rand(10,3)
class Wine(Dataset):
    def __init__(self,X):
        self.x = X[:, 1:]
        self.y = X[:, 0]
        self.num_samples = X.shape[0]

    def __len__(self):
        return(self.num_samples)

    def __getitem__(self, index):
        x = torch.tensor(self.x[index] , dtype= torch.float32)
        y = torch.tensor(self.y[index] , dtype= torch.float32) 
        return x, y
#        
WineDS = Wine(X)
#
print('size of train ds ',len(WineDS))
train_dataloader = DataLoader(WineDS, batch_size=2, shuffle=False)
#
train_features , train_labels = next(iter(train_dataloader))
print(train_features)
print(train_features.dtype)
print('----')
print(train_labels)
print(train_labels.dtype)
</code></pre>
<p>// size of train ds  10<br>
// tensor([[0.7028, 0.4137],<br>
//         [0.7279, 0.1905]])<br>
// torch.float32<br>
// ----<br>
//  tensor([0.1961, 0.5768])<br>
// torch.float32</p>