akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Error with torch on jetson

Mr.Panda: <p>I am using pytorch on a jetson device. Here are the versions:<br>
torch: 2.1.0a0+41361538.nv23.06<br>
torchvision: 0.16.1</p>
<p>I am using this piece of code to resize my images.</p>
<pre><code class="lang-auto">transform = v2.Compose([
    v2.Resize((224, 224), interpolation=v2.InterpolationMode.BILINEAR)
])
</code></pre>
<p>I am getting the following error:<br>
AttributeError: module ‘torch.backends’ has no attribute ‘cpu’</p>
<p>Can someone help me fix this? Thanks a lot</p>

ptrblck: <p>Your <code>torchvision</code> release might be too new so build an older and compatible one.</p>

Mr.Panda: <p>Okay, thank you, I will try building 0.15.</p>
Converting GPT-2 to TorchScript

anAnnoyingNerd: <p>I’ve been having trouble converting a GPT-2 model to TorchScript. I have been able to successfully convert the model, but the data it outputs isn’t anywhere similar to the original model. For example, I converted the model to TorchScript with the sample input “A compound sentence is”. The original model outputs something like A compound sentence is a sentence that is not a sentence at all." The TorchScript model just outputs random words that have nothing to do with the input. Can somebody help me figure out how I can fix this?</p>

ptrblck: <p>I don’t know if you are using tracing with data-dependent control flow in your model or scripting, as at least the former workflow would create invalid results since conditions and code paths are baked into the model.<br>
In any case, <code>TorchScript</code> is in maintenance mode and won’t get any major features anymore.<br>
Did you try the newly introduced <code>torch.compile</code> mode instead?</p>

fabian_schutze: <p>In addition to what, <a class="mention" href="/u/ptrblck">@ptrblck</a>  said, I usually slice the models into different parts and then start with jiting the first, first and second, etc …, part to see at which stage jitting results diverge if deploying the whole model does not work immediately.</p>

anAnnoyingNerd: <p>I’ll have to set up WSL, because Windows isn’t supported by <code>torch.compile()</code> yet.</p>

anAnnoyingNerd: <p>Could you explain more?</p>

Mr_young: <p>hi, can you show your code for converting a GPT-2 model to TorchScript?</p>
How does pytorch express tensor operations in terms of BLAS routines?

valya: <p>Almost any kind of tensor operation (in particular, tensor contraction) in pytorch is comparable in speed to a matrix multiplication of similar size, so I’m assuming that pytorch is able to express almost any sort of tensor operation in terms of BLAS routines. My goal is to understand how exactly pytorch does that.</p>
<p>I can work out particular cases by hand, but I’m not sure how to generalise that to any arbitarary tensor contraction. Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
AttributeError: 'collections.OrderedDict' object has no attribute 'to'

111153: <p>The following is my test_model.py. When executing it, there exist AttributeError: ‘collections.OrderedDict’ object has no attribute ‘to’.   Pytorch version is 1.2.0      python version 3.7.4<br>
import torch</p>
<p>from PIL import Image<br>
from torchvision import transforms</p>
<p>classes = (‘a’,‘b’,‘c’,‘d’)<br>
device = torch.device(‘cuda’)<br>
transform=transforms.Compose([<br>
transforms.Resize(256),<br>
transforms.CenterCrop(224),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485,0.456,0.406],<br>
std=[0.229,0.224,0.225])<br>
])<br>
def prediect(img_path):<br>
net=torch.load(‘best.mdl’)<br>
net=net.to(device)<br>
torch.no_grad()<br>
img=Image.open(img_path)<br>
img=transform(img).unsqueeze(0)<br>
img_ = img.to(device)<br>
outputs = net(img_)<br>
_, predicted = torch.max(outputs, 1)<br>
# print(predicted)<br>
print(‘this picture maybe :’,classes[predicted[0]])<br>
if <strong>name</strong> == ‘<strong>main</strong>’:<br>
prediect(’./test/name.jpg’)</p>
<hr>
<p>runfile(‘C:/classification-master/Test_model.py’, wdir=‘C:/classification-master’)<br>
Traceback (most recent call last):</p>
<p>File “”, line 1, in <br>
runfile(‘C:/classification-master/Test_model.py’, wdir=‘C:/classification-master’)</p>
<p>File “C:\Users\derek\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py”, line 827, in runfile<br>
execfile(filename, namespace)</p>
<p>File “C:\Users\derek\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py”, line 110, in execfile<br>
exec(compile(f.read(), filename, ‘exec’), namespace)</p>
<p>File “C:/classification-master/Test_model.py”, line 27, in <br>
prediect(’./test/name.jpg’)</p>
<p>File “C:/classification-master/Test_model.py”, line 17, in prediect<br>
net=net.to(device)</p>
<p>AttributeError: ‘collections.OrderedDict’ object has no attribute ‘to’</p>

JuanFMontesinos: <p>When you do <code>net=torch.load(...)</code> you are loading pretrained weights. It is just a python dictionary containing the weights, not the pytorch network. You need (in addition) to set a model (<code>model= Model(*args,**kwargs)</code> and then to load those weights into the model (<code>model.load_state_dict(net)</code>)</p>

Euridy_Libritannia: <p>Thanks a lot Juan. Your solution worked, at least for me.<br>
I was trying to load a pkl GAN generator, so I did as you said and it works!<br>
I also changed net.to(device) to model.to(device), for information.</p>
Encrypt/decrypt jit model in c++

Steve25: <p>I have a model I trained in pytorch. I have exported it using torch.jit.trace() and have a model.pt file. In c++ (libtorch) I am able to load and predict using this mode.l I want to be able to encrypt the model.pt file. Decrypt it in memory, make predictions. How can I do this. Not sure how to load a jit trace model from memory or how to encrypt it.</p>

tom: <p>Last I looked, you could use std::stringstream to load from memory. Or you could look for an encryption library that provides an iostream interface.</p>
<p>Best regards</p>
<p>Thomas</p>

GabrielB: <p>Doing this is useless because dumping a programs memory takes 5 seconds. If you don’t want people to have the model then don’t put it on their computer</p>
NCPS, Liquid Neural Network Example

pumplerod: <p>I’ve been trying to learn more about Liquid Neural Networks and their application with respect to time series data.  I’ve found the <a href="https://ncps.readthedocs.io/en/latest/api/torch.html" rel="noopener nofollow ugc">ncps library</a> and am struggling a bit to translate their very basic sine wave example into use with time series data such as financial market prices or inventory/sales data.  I’m wondering if there is anyone who might be able to provide a little more help/insight.</p>
<p>I’m able to create a Neural Circuit Policy and  LTC model from their example, however the training and implementation with respect to a more realistic dataset is not obvious to me.  I can’t seem to locate any concrete examples of this on-line and, unfortunately, the <a href="https://arxiv.org/abs/2006.04439" rel="noopener nofollow ugc">original paper</a> while fascinating, is beyond my grasp when it comes to translating it into a pytorch model.</p>
<p>As a very basic example, I’m attempting to take the closing price of a security and predict its future value n days forward in time.  When following some other similar examples such as using a LSTM model I am able to train and get results, but fear I’m not actually implementing the model correctly.</p>
<p>per the example in their documentation if I provide a time window of say 30 days, I actually get a prediction output which is also 30 days worth of data, and while I can supply a target for each day I’m really only looking for the prediction those past 30 days will provide.  Simply taking the last index of the prediction doesn’t really feel to me like I’m properly using this model.</p>
<p>Could anyone point me in the direction of a more representative example, or be willing to connect and expand my understanding of the proper usage?</p>
Transformer Encoder changes behavior with @torch.no_grad()

JLenz: <p>Hello,</p>
<p>I am trying to investigate some irregular behaviors in my trained network. The encoder is a standard pytorch Transformer encoder. I have identified that, when adding the <code>@torch.no_grad()</code> decorator to a given function, the encoder outputs will change each time. For example:</p>
<pre><code class="lang-auto">def forward_1(self, x):
return self.encoder(x)

@torch.no_grad()
def forward_2(self, x):
return self.encoder(x)
</code></pre>
<p>If I set the model to <code>eval()</code> and run both functions above with an identical input, I will get to different outputs.</p>
<p>If I am not mistaken, with <code>model.eval()</code> all differentiating factors like dropout should be removed, and gradient computation itself should not be in any way affecting the outputs. In other words, <strong>shouldn’t the two functions above return the exact same Tensor?</strong></p>
<p>Many thanks in advance.</p>

soulitzer: <p>It is expected that the numerics are different in different code paths, depending on the requires grad-ness of the inputs the encoder may dispatch to different sdpa backends.</p>
What to do if your merged PR got reverted?

adrianwaelchli: <p>Sometimes, the PyTorch team reverts a commit after it has been reviewed and merged into the main branch. This is fair, since they may have internal tests or find other issues with the PR that need it reverted. But for the open source community, it is not possible to understand what the issue was, or what the next step is, because the reverted commit does not contain any information about what went wrong, only what appears to be Meta-internal ticket numbers.</p>
<p>My suggestions:</p>
<ol>
<li>Provide the error stack trace and possibly the entire test case that failed.</li>
<li>Reopen the tickets that were closed in the original PR. Only PyTorch staff can do this.</li>
<li>Communicate with the author, encourage them to re-land an improved version of the reverted commit. Guide them with clear call to action of what should be changed.</li>
</ol>
<p>I’m posting here, seeking advice because my PR got <a href="https://github.com/pytorch/pytorch/commit/199b04fdbd69e9a27edf676fd10e21bbd2a8d416" rel="noopener nofollow ugc">reverted</a> and I have no idea what to do.</p>

nairbv: <p>I think the suggestions you made are supposed to be standard practice, and that there are no relevant comments on your original PR seems like a mistake. You might want to add a comment on your original PR pinging internal people to ask about next steps before re-submitting.</p>
<p>Another thing to check is the hud though. Often when a PR is reverted, you’ll see the test failure in the HUD right before that revert. In your case, if step through to the page with the revert for your PR, I see a whole series of PR unlands happening, including yours. It’s possible that your PR merely had merge conflicts with the PR they actually cared to revert, and so it was easier to revert everything until back to a known working state.</p>
<p>If your PR was directly related to what actually broke things, I think they’d have been less likely to forget to add those details to your original PR. In at least one case, looking at the many unlanded PRs just before yours, I see some have comments like “It is required to revert this PR in order to properly revert <span class="hashtag-raw">#116193</span>.” Some of the other nearby unlands have comments with specific references to particular test failures.</p>
<p>The hud link (though the correct page number may change as more commits come in):<br>
<a href="https://hud.pytorch.org/hud/pytorch/pytorch/main/3?per_page=50" class="onebox" target="_blank" rel="noopener nofollow ugc">https://hud.pytorch.org/hud/pytorch/pytorch/main/3?per_page=50</a></p>
Modifying convolution weights in the forward pass

RohitDulam: <p>I would like to modify the weights of a convolution operation before the convolution operation on the input. For example -</p>
<pre><code class="lang-auto">conv = torch.nn.Conv2d(in_ch, out_ch, 3, 1, 1)
conv.weight.data = conv.weight * values
out = conv(x)
</code></pre>
<p>In the above code, ‘x’ is the input and the convolutional weights are modified using the ‘.data’ attribute. ‘values’ is a tensor of the size of kernel_size x kernel_size which in the above example is 3 x 3. I’ve read from multiple sources how using the ‘.data’ attribute is deprecated and shouldn’t be used since it leads to incorrect gradient calculations. I would like to know the right way to modify the convolutional weights.</p>

ptrblck: <p>Wrap the weight manipulation into a <code>with torch.no_grad()</code> context to explicitly disable Autograd for this operation instead of manipulating the <code>.data</code> attribute.</p>

RohitDulam: <p>Thank you for your reply <a class="mention" href="/u/ptrblck">@ptrblck</a>. What if ‘values’ are being generated by another sub-network and thus gradient flow is necessary through ‘values’ as well? In that case, will <code>torch.no_grad()</code> work?</p>

ptrblck: <p>No, it won’t work. In this case you might want to use the functional API via:</p>
<pre data-code-wrap="python"><code class="lang-python">out = F.conv2d(x, conv.weight*values, ...)
</code></pre>
<p>instead of manipulating the <code>weight</code> of the conv layer.</p>

RohitDulam: <p>Ok, I was wondering the same (to use the functional API). Seems like there’s no way to use it in the nn.Conv2d formulation anymore. Thank you.</p>
In DDP mode, how to increase the number of negative samples in contrast learning?

skye95git: <p>I’m using contrastive learning to train categorization models. I tried to get better performance by increasing the batch size. Each GPU can process 64 batches at most, so I changed the number of Gpus from 4 to 16 and tried to increase the batch size from 256 to 1024. However, it was found in the experiment that increasing the number of Gpus could not increase the number of negative samples in a batch, because the batch processed by each GPU is always 64. For each GPU, the number of negative samples is always 63, which has nothing to do with the number of Gpus.<br>
In DDP mode, how can we increase the number of negative samples?<br>
For example, when the total batch size changes to 1024, the negative sample number should be 1023 instead of 63</p>

anj: <p>Can you try increasing the batch size to 1024 with 4 GPUs which should increase batch size/GPU to 256? That would increase the number of samples seen by each GPU. Is that what you are trying to accomplish?</p>
<p>cc <a class="mention" href="/u/agu">@agu</a> <a class="mention" href="/u/rvarm1">@rvarm1</a></p>

zhenxz0121: <p><a href="https://github.com/zsnoob/EfficientDDP-4-Contrastive-Train" rel="noopener nofollow ugc">zsnoob/EfficientDDP-4-Contrastive-Train: Optimizing the way of contrastive learning in PyTorch-DDP(DistributedDataParallel) multi-GPU training (github.com)</a> There is a github repository about that question, and some optimization of your concern about “all machine are doing the same thing in the loss calculation”. Let’s discuss about it.</p>
Torch.distributed.all_gather() to compute Noise Contrastive Loss with PyTorch

joohyunglee: <p>Computing infoNCE requires gathering all encoded representations from all GPUs for full negative sampling. Specifically, to compute infoNCE loss, many repositories, e.g. <a href="https://github.com/Spijkervet/SimCLR/blob/master/simclr/modules/nt_xent.py#L36" rel="noopener nofollow ugc">SimCLR</a>, <a href="https://github.com/rdangovs/essl/blob/main/imagenet/simclr/main.py#L232" rel="noopener nofollow ugc">essl</a>, uses 1) <code>torch.distributed.all_gather()</code> to gather features from all GPUs at <code>forward()</code> and 2) <code>torch.distributed.all_reduce()</code> to sum the gradients at <code>backward()</code>.</p>
<p>However, wouldn’t the synchronized representations (using <code>torch.distributed.all_gather()</code>) make all GPUs compute the exact same loss at all GPUs and are thus redundant?</p>
<p>How about sending all features to a single machine (either GPU or CPU), computing the loss there, and multiplying the gradient by the # of GPUs? <a href="https://pytorch.org/docs/stable/notes/ddp.html" rel="noopener nofollow ugc">because gradient will be averaged throughout GPUs</a></p>

kwen2501: <aside class="quote no-group" data-username="joohyunglee" data-post="1" data-topic="162015">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/joohyunglee/48/12291_2.png" class="avatar"> joohyunglee:</div>
<blockquote>
<p>However, wouldn’t the synchronized representations (using <code>torch.distributed.all_gather()</code>) make all GPUs compute the exact same loss at all GPUs and are thus redundant?</p>
</blockquote>
</aside>
<p>That’s a fair concern. I don’t know if the program uses DDP. With DDP, since the data input to each GPU would be different, maybe the loss computed at each GPU would be different? And hence a need to all-reduce the gradients?</p>

joohyunglee: <p>Input to GPUs are obviously different. However, i think the representation from the encoder becomes the same due to <code>all_reduce()</code>(most infoNCE self-sup repositories synchronize the representation to use all representation for the negative sample). Therefore, i assume gathering all representation to a single machine, eg gpu or cpu, may reduce the redundant loss computation.</p>
<p>And my question is if my concern is correct or not. Thank you!</p>

xuef: <p>I would agree with the concern. This can be done a little differently such that each GPU only calculate the loss for its own batch after all_gather, that way each GPU is doing similar computing, and gradient reduce would carry to all GPUs.</p>

zhenxz0121: <p><a href="https://github.com/zsnoob/EfficientDDP-4-Contrastive-Train" rel="noopener nofollow ugc">zsnoob/EfficientDDP-4-Contrastive-Train: Optimizing the way of contrastive learning in PyTorch-DDP(DistributedDataParallel) multi-GPU training (github.com)</a> There is a github repository about that question, and some optimization of your concern about “all machine are doing the same thing in the loss calculation”. Let’s discuss about it.</p>
How to update stduent network's some layers with gradient and other layers with moving average in DDP?

ShanakaRG: <p>I have a student-teacher network for a self-supervised learning network. Basically, my teacher network predicts classes and calculates the loss and updates its parameters using the gradient flow. Typically, the student network updates its parameters using the moving average of teacher network. But in my case, i have some unique layers for the student network that are needed to train using gradient flow.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/2/22e11ca7182e82ec88f90cac29ca4c00bf007062.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/22e11ca7182e82ec88f90cac29ca4c00bf007062" title="network"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22e11ca7182e82ec88f90cac29ca4c00bf007062_2_666x500.jpeg" alt="network" data-base62-sha1="4Yyx6XIsU6Trffbh76gs6Cn0X9o" width="666" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22e11ca7182e82ec88f90cac29ca4c00bf007062_2_666x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22e11ca7182e82ec88f90cac29ca4c00bf007062_2_999x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22e11ca7182e82ec88f90cac29ca4c00bf007062_2_1332x1000.jpeg 2x" data-dominant-color="B5B2A4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">network</span><span class="informations">2048×1536 356 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>For instance, as the figure shows, there is a unique layer (layer X) in the student network, and rest are common for both networks. The layer 1 layer 2 and layer 3 in the student network need to be updated with the moving average of teacher network and the layer x needs to be updated with gradient flow. How do I design such a network using Python torch DDP.</p>
Returning data of different lengths

Debajyoti_Sengupta: <p>Hello all,</p>
<p>I am trying to figure out a solution for my use case where I have to return three objects from my dataset that are of different lengths. Here’s a short snippet that explains what I have now.</p>
<pre><code class="lang-auto">class MyDataset(Dataset):
    def __init__(
        self,
        data_dir: list[Path],
    ) -&gt; None:
        """
        Args
        ----
        data_dir: list[Path]
            List of paths to data npy files.
        """
        self.data_dir = data_dir
        data, data2, labels = load_data(self.data_dir)
        # here, data and labels are of the same length
        # data2 is of a different length.
        self.num_len = min(len(data), len(data2))

    def __len__(self) -&gt; int:
        return self.num_len
    
    def __getitem__(self, idx) -&gt; np.ndarray:
            return self.data[idx], self.labels[idx], self.data2[idx]
</code></pre>
<p>My model requires <code>self.data</code> during the training stage; <code>self.data</code> and <code>self.data2</code> in the validation; and <code>self.data</code>, <code>self.labels</code> during the prediction stage.</p>
<p>To keep things running, I am taking the minimum of the two and return that as the length of the dataset, but that means <strong>a lot</strong> of the data remains unused (?).<br>
I was wondering if there is a smart, more efficient way to deal with this?</p>
About initialization of RecursiveScriptModule instance?

tuanhm2302: <p>I have taken a look at <strong>Torchscript</strong> source code and in the <strong>“_construct” method</strong> of class <strong>RecursiveScriptModule (<a href="https://github.com/pytorch/pytorch/blob/main/torch/jit/_script.py#L632" rel="noopener nofollow ugc">torch/jit/_script.py</a>)</strong> there are some guiding comment lines that i dont really understand it.</p>
<p><em>PyTorch code should use this to construct a RecursiveScriptModule instead</em><br>
<em>of calling <code>__init__</code> directly, as it makes sure the</em><br>
<em>object is properly finalized (and in the future, we may take</em><br>
<em>control of how the RecursiveScriptModule instance is created).</em></p>
<p>Why <strong>_construct</strong> method can help to control the instance initialization instead of calling <em><strong>__init __</strong></em> directly? Can someone explain and give me some examples for this?</p>
COSINEEMBEDDINGLOSS returns much higher loss on GPU

Jethro_Low: <p>Hi there,</p>
<p>I am training a quantization aware model to output an embedding of size (677, 1408) to be as similar as possible to the original embedding. For this, I am using COSINEEMBEDDINGLOSS as the loss function. When i run my training script on cpu, it works fine and the loss is ~ 0.13 per batch. However, when I run the same training script on the GPU, the loss becomes ~ 1.04 per batch and this leads to error. RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasLtMatmul with transpose_mat1 1 transpose_mat2.</p>
<p>How can this be resolved? And is the issue related to the COSINEEMBEDDINGLOSS?</p>
<p>Thank you!</p>

ptrblck: <p>Could you post a minimal and executable code snippet reproducing the error in the latest PyTorch release, please?</p>

Jethro_Low: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> ,</p>
<p>Sorry for the delayed reply. I found out the issue. It has nothing to do with the cosineembeddingloss actually. The issue was with the mismatch between the torch and torchvision versions and the CUDA versions. When installing some libraries for the blip2 model, the torch and torchvision versions got updated as well. I installed some of these libraries with --no-deps during the pip install and now it is working fine.</p>
<p>Thank you for your help!</p>
Run TensorFlow model in Torchserve

OkabeSan: <p>Hi, I read the Torchserve documentation but didn’t find my answer.<br>
Can we run any type of models and frameworks like TensorFlow and etc. in Torchserve?<br>
or I should convert to ONNX format and write the <code>Inference</code> function?</p>

Wen_Flybywind: <p>Hi, <a class="mention" href="/u/okabesan">@OkabeSan</a> I got the same idea with u. In this sense, torchserve can be regarded as a general MLserve platform. Have u got any practice/experience on it ? Glad if u can share something!</p>
Could not run 'torchvision::nms'

masoud_alishahi: <p>Hello,<br>
I am using Windows 11, and I encounter an error with the YOLOv8 model in the following line:<br>
results = model.track(frame, persist=True, conf=0.6)</p>
<p>The error is:<br>
Could not run ‘torchvision::nms’ with arguments from the ‘CUDA’ backend</p>
<p>nvcc --version<br>
nvcc: NVIDIA (R) Cuda compiler driver<br>
Copyright (c) 2005-2023 NVIDIA Corporation<br>
Built on Wed_Feb__8_05:53:42_Coordinated_Universal_Time_2023<br>
Cuda compilation tools, release 12.1, V12.1.66<br>
Build cuda_12.1.r12.1/compiler.32415258_0</p>
<p>torch                   2.1.2+cu121<br>
torchaudio              2.1.2+cu121<br>
torchvision             0.16.2</p>
<p>Please provide detailed guidance on how to resolve this issue.</p>

ptrblck: <p>You might need to install <code>torchvision</code> with CUDA support as it seems the CPU-only version was installed.</p>

masoud_alishahi: <p>pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url <a href="https://download.pytorch.org/whl/cu118" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu118</a></p>
<p>The problem was solved by using the above command. Thank you very much.</p>
How do I input a multivariate time series data properly?

Jordan_Howell1: <p>Hello,<br>
I’m new to pytorch and would like to experiment on a timeseries forecasting problem.  My training data consists monthly sales data, a three month moving average, as well as a flag denoting if a sales promotion is happening or not.  I am trying to forecast for the next 12 months out.</p>
<p>My training data consists of 37 observations, six lags of all three features with size <code>(37, 6, 3)</code>.  My target data set is the next 12 months of sales data with size <code>(37, 12)</code>.</p>
<p>When I try to convert my arrays into a torch.tensor using the below code, I get the below error:</p>
<pre><code class="lang-auto">Xtrain_val_tens = torch.tensor(Xtrain_val, dtype=torch.float64)
Ytrain_val_tens = torch.tensor(Ytrain_val, dtype = torch.float64)
</code></pre>
<p>ERROR</p>
<pre><code class="lang-auto">TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool.
</code></pre>
<p>I tried the below with no luck:</p>
<pre><code class="lang-auto">Xtrain_val_tens = torch.from_numpy(Xtrain_val)
Ytrain_val_tens = torch.from_numpy(Ytrain_val)
</code></pre>
<p>If it helps, I start with a dataframe and put said df through the following code to get my arrays:</p>
<pre><code class="lang-auto">lags = 6
horizon = 12
###############Make a multi-output time series dataset#####################
series = samp_df[['ord_qty', '3_day_moving_avg', 'promotion_flag']]
#number of lags in the input
Tx = lags
#number of steps in the output 
Ty = horizon
#instantiate list to take in the train (X) and test(Y) data set
X = []
Y = []
# run a series from 0 to the length of the dataframe minus the horizon to create a "step through time" data set
for t in range(len(series) - Tx - Ty + 1):
    x = series[t:t+Tx]
    X.append(x)
    y = series['ord_qty'][t+Tx:t+Tx+Ty]
    Y.append(y)
# #reshape into numpy arrays for faster processing in ML models
X = np.array(X)
Y = np.array(Y).reshape(-1, Ty)
Xtrain_val, Ytrain_val = X[:-1], Y[:-1]
Xtest_val, Ytest_val = X[-1:], Y[-1:]
</code></pre>

Jordan_Howell1: <p>I just figured this out. I needed to make the numpy array a float64 by adding <code>np.array(X).astype('float64')</code> onto the arrays I was using.</p>