Runtime Error : The size of tensor a (368) must match the size of tensor b (16) at non-singleton dimension 0

Bonseung_Koo: <p>Hi, professionals. I’m trying to use PatchTST Model(Multivariate Timeseries forecasting using Transformers). I found out “tsai” which makes using PatchTST model easier, so I try to use tsai module in my project.</p>
<h2><a name="this-is-my-x-and-y-shape-38874-23-6-38874-1-1-i-use-23-columns-to-predict-one-columns-my-dataframe-has-38874-rows-1" class="anchor" href="#this-is-my-x-and-y-shape-38874-23-6-38874-1-1-i-use-23-columns-to-predict-one-columns-my-dataframe-has-38874-rows-1"></a>This is my X and y shape. ((38874, 23, 6), (38874, 1, 1))<br>
I use 23 columns to predict one columns. My dataframe has 38874 rows.</h2>
<p>This is my initial hyperparameter.<br>
arch_config = dict(<br>
n_layers=3,  # number of encoder layers<br>
n_heads=4,  # number of heads<br>
d_model=16,  # dimension of model<br>
d_ff=128,  # dimension of fully connected network<br>
attn_dropout=0.0, # dropout applied to the attention weights<br>
dropout=0.3,  # dropout applied to all linear layers in the encoder except q,k&amp;v projections<br>
patch_len=4,  # length of the patch applied to the time series to create patches<br>
stride=2,  # stride used when creating patches<br>
padding_patch=True,  # padding_patch)</p>
<hr>
<h1><a name="this-is-my-model-architecture-patchtst-input-shape-16-x-23-x-6-2" class="anchor" href="#this-is-my-model-architecture-patchtst-input-shape-16-x-23-x-6-2"></a>This is my model architecture.<br>
PatchTST (Input shape: 16 x 23 x 6)</h1>
<h1><a name="layer-type-output-shape-param-trainable-3" class="anchor" href="#layer-type-output-shape-param-trainable-3"></a>Layer (type)         Output Shape         Param #    Trainable</h1>
<pre><code>                 16 x 23 x 1         
</code></pre>
<p>RevIN                                     46         True</p>
<hr>
<pre><code>                 16 x 23 x 8         
</code></pre>
<p>ReplicationPad1d</p>
<hr>
<pre><code>                 16 x 4 x 3          
</code></pre>
<p>Unfold</p>
<hr>
<pre><code>                 16 x 23 x 3 x 16    
</code></pre>
<p>Linear                                    80         True<br>
Dropout<br>
Linear                                    272        True<br>
Linear                                    272        True<br>
Linear                                    272        True<br>
Dropout<br>
Linear                                    272        True<br>
Dropout<br>
Dropout</p>
<hr>
<pre><code>                 16 x 16 x 3         
</code></pre>
<p>Transpose<br>
BatchNorm1d                               32         True</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Transpose</p>
<hr>
<pre><code>                 16 x 3 x 128        
</code></pre>
<p>Linear                                    2176       True<br>
GELU<br>
Dropout</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Linear                                    2064       True<br>
Dropout</p>
<hr>
<pre><code>                 16 x 16 x 3         
</code></pre>
<p>Transpose<br>
BatchNorm1d                               32         True</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Transpose<br>
Linear                                    272        True<br>
Linear                                    272        True<br>
Linear                                    272        True<br>
Dropout<br>
Linear                                    272        True<br>
Dropout<br>
Dropout</p>
<hr>
<pre><code>                 16 x 16 x 3         
</code></pre>
<p>Transpose<br>
BatchNorm1d                               32         True</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Transpose</p>
<hr>
<pre><code>                 16 x 3 x 128        
</code></pre>
<p>Linear                                    2176       True<br>
GELU<br>
Dropout</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Linear                                    2064       True<br>
Dropout</p>
<hr>
<pre><code>                 16 x 16 x 3         
</code></pre>
<p>Transpose<br>
BatchNorm1d                               32         True</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Transpose<br>
Linear                                    272        True<br>
Linear                                    272        True<br>
Linear                                    272        True<br>
Dropout<br>
Linear                                    272        True<br>
Dropout<br>
Dropout</p>
<hr>
<pre><code>                 16 x 16 x 3         
</code></pre>
<p>Transpose<br>
BatchNorm1d                               32         True</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Transpose</p>
<hr>
<pre><code>                 16 x 3 x 128        
</code></pre>
<p>Linear                                    2176       True<br>
GELU<br>
Dropout</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Linear                                    2064       True<br>
Dropout</p>
<hr>
<pre><code>                 16 x 16 x 3         
</code></pre>
<p>Transpose<br>
BatchNorm1d                               32         True</p>
<hr>
<pre><code>                 16 x 3 x 16         
</code></pre>
<p>Transpose</p>
<hr>
<pre><code>                 16 x 23 x 48        
</code></pre>
<p>Flatten</p>
<hr>
<pre><code>                 16 x 23 x 1         
</code></pre>
<p>Linear                                    49         True</p>
<hr>
<p>Total params: 16,351<br>
Total trainable params: 16,351</p>
<p>This is my running function and starting code.<br>
learn = TSForecaster(X, y, splits=splits, batch_size=16, path=“models”,<br>
arch=“PatchTST”, arch_config=arch_config,<br>
metrics=[r2_score, mae, rmse], cbs=ShowGraph())</p>
<p>n_epochs = 100<br>
<span class="hashtag-raw">#lr_max</span> = learn.lr_find().valley<br>
lr_max = 0.0025<br>
learn.fit_one_cycle(n_epochs, lr_max=lr_max)<br>
learn.export(‘patchTST.pt’)</p>
<p>Why I have errors in mismatch tensor size?!! I don’t know where tensor number 368 and 16 came from. Does my hyper parameter and data size are mismatching at first? I really can’t find out why…<br>
Please help me!!</p>

ptrblck: <p>Your code is unfortunately not properly formatted and contains seemingly random values without any explanation what these refer to, so I’m also unsure where the mismatch is coming from.</p>

Bonseung_Koo: <p>Hi I’m Koo from South Korea, currently majoring in Industrial Engineering in Korea University. First of all, I really appreciate your feedback from my question.<br>
Question that I posted was from multivariate timeseries forecasting problem, using transformer(patchTST).<br>
My objective was to predict plant’s temperature(temperature sensor in market), using various variables in X label.</p>
<p>Since I realize that you are so professional and intelligent in Pytorch, I wish I can hear more about my problem.<br>
I attached my ipynb file and my dataset, which I try to use in the ipynb file.<br>
Ipynb file’s lines are not much since it uses “tsai” module which contains original patchTST model from official github.<br>
Would you like to give me a favor of looking through my codes and dataset?</p>
<p>If you can’t I still appreciate your help and works you’ve made in this site.<br>
Have a nice day and take care.</p>
<p>Best regards<br>
Koo</p>
<p>2023년 12월 15일 (금) 오후 11:56, ptrblck via PyTorch Forums &lt;<a href="mailto:noreply@discuss.pytorch.org">noreply@discuss.pytorch.org</a>&gt;님이 작성:</p>
<p>(Attachment PatchTST_Practice.ipynb is missing)</p>
<p>(Attachment airconditioner.csv is missing)</p>

emanuelshalev: <p>In the working TSAI PatchTST tutorial, the predicted features are the same number as the input features.<br>
I have a similar case to yours where I want to predict a subset of the input timeseries features.<br>
The error is pointing inside TSAI code, it looks like they are not expecting such case.</p>
<pre><code class="lang-auto">File ~\.conda\envs\tsai\lib\site-packages\torch\nn\functional.py:3315, in mse_loss(input, target, size_average, reduce, reduction)
   3308 r"""mse_loss(input, target, size_average=None, reduce=None, reduction='mean') -&gt; Tensor
   3309 
   3310 Measures the element-wise mean squared error.
   3311 
   3312 See :class:`~torch.nn.MSELoss` for details.
   3313 """
   3314 if has_torch_function_variadic(input, target):
-&gt; 3315     return handle_torch_function(
   3316         mse_loss, (input, target), input, target, size_average=size_average, reduce=reduce, reduction=reduction
   3317     )
   3318 if not (target.size() == input.size()):
   3319     warnings.warn(
   3320         f"Using a target size ({target.size()}) that is different to the input size ({input.size()}). "
   3321         "This will likely lead to incorrect results due to broadcasting. "
   3322         "Please ensure they have the same size.",
   3323         stacklevel=2,
   3324     )
</code></pre>
<p>My impression is that changing TSAI code would be required to make it work, correcting the input target to filter out unused features to calculate MSELoss against the target features. That been said even if that works they suggest that could lead to incorrect results.</p>
<p>Anyone else has more insights here?</p>