TemporalFusionTransformer passing whole module to torch.empty()

Loggy: <p>torch version 2.0.1+cu117<br>
pytorch_forecasting version 1.0.0<br>
pytorch_lightning version 2.0.7<br>
[Using the CPU for testing]</p>
<p>I am  trying to set up a TFT model but I get the following error traceback:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “ShortTrain2.py”, line 105, in <br>
model = TFT(<br>
File “/home/jl/.local/lib/python3.8/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/<strong>init</strong>.py”, line 248, in <strong>init</strong><br>
self.static_context_variable_selection = GatedResidualNetwork(<br>
File “/home/jl/.local/lib/python3.8/site-packages/pytorch_forecasting/models/temporal_fusion_transformer/sub_modules.py”, line 205, in <strong>init</strong><br>
self.fc1 = nn.Linear(self.input_size, self.hidden_size)<br>
File “/home/jl/.local/lib/python3.8/site-packages/torch/nn/modules/linear.py”, line 97, in <strong>init</strong><br>
self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))<br>
TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:</p>
<ul>
<li>(tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)</li>
<li>(tuple of ints size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)</li>
</ul>
</blockquote>
<p>I’ve traced this back to the line in the TFT <strong>init</strong>.py in the static encoder section where the input_size, hidden_size and output_size are all set to self.hparams.hidden_size but instead of a size tuple or whatever being passed, the object passed is the whole TimeSeriesDataSet module with a length parameter.</p>
<p>I’m sure this is an input issue with a misunderstanding but I would welcome some direction. Here is the code:</p>
<p>import os<br>
import sys<br>
import logging<br>
import math<br>
import warnings<br>
import pandas as pd<br>
import numpy as np<br>
import torch<br>
import pytorch_lightning as pl<br>
import pytorch_forecasting as pf<br>
from pytorch_lightning import Trainer<br>
from pytorch_lightning.callbacks import EarlyStopping,LearningRateMonitor<br>
from pytorch_lightning.loggers import TensorBoardLogger<br>
from pytorch_forecasting import Baseline,TimeSeriesDataSet<br>
from pytorch_forecasting.models import TemporalFusionTransformer as TFT<br>
from pytorch_forecasting.data import MultiNormalizer,TorchNormalizer<br>
print(“python version”,sys.version)<br>
print(“pandas version”,pd.<strong>version</strong>)<br>
print(“numpy version”,np.<strong>version</strong>)<br>
print(“torch version”,torch.<strong>version</strong>)<br>
print(“pytorch_forecasting version”,pf.<strong>version</strong>)<br>
print(“pytorch_lightning version”,pl.<strong>version</strong>)<br>
pd.options.display.max_rows = 100<br>
os.environ[“CUDA_VISIBLE_DEVICES”] = “”<br>
logging.getLogger(“pytorch_lightning”).setLevel(logging.WARNING)<br>
Train=pd.read_csv(“ShortTrain2.csv”)<br>
Train[Train.select_dtypes(include=[‘float64’]).columns] = Train[Train.select_dtypes(include=[‘float64’]).columns].astype(‘float32’)<br>
aminx=<span class="chcklst-box fa fa-square-o fa-fw"></span><br>
for i in Train.minx:<br>
aminx.append(“A”+str(math.floor(i/12)))<br>
aminx=pd.DataFrame(aminx)<br>
aminx.columns=[‘aminx’]<br>
Train=pd.concat((Train,aminx),axis=1)<br>
print(1,Train.dtypes)<br>
predictors=1<br>
if len(sys.argv) &gt; 1 :<br>
predictors = 2<br>
xtarget,xtarget_normalizer,xtime_varying_known_reals,xtime_varying_unkown_reals=None,None,None,None<br>
xtime_varying_known_categoricals=[‘aminx’]<br>
if predictors == 1:<br>
print(“Using a single predictor v1_R”)</p>
<h1><a name="single-predictor-1" class="anchor" href="#single-predictor-1"></a>single predictor</h1>
<pre><code>xtarget=['v1_R']
xtarget_normalizer=MultiNormalizer([TorchNormalizer()])
xtime_varying_known_reals=['v1_Act']
xtime_varying_unknown_reals=['v1_C','v1_D','v1_V','v1_S','v1_H','v1_L','v1_R_lag10','v1_R_lag20']
</code></pre>
<p>else:<br>
print(“Using 2 predictors v1_R and v2_R”)</p>
<h1><a name="h-2-predictors-2" class="anchor" href="#h-2-predictors-2"></a>2 predictors</h1>
<pre><code>xtarget=['v1_R','v2_R']
xtarget_normalizer=MultiNormalizer([TorchNormalizer(),TorchNormalizer()])
xtime_varying_known_reals=['v1_Act','v2_Act']
xtime_varying_unknown_reals=['v1_C','v1_D','v1_V','v1_S','v1_H','v1_L','v1_R_lag10','v1_R_lag20','v2_C','v2_D','v2_V','v2_S','v2_H','v2_L','v2_R_lag10','v2_R_lag20']
</code></pre>
<p>print(xtarget)<br>
print(xtarget_normalizer)<br>
print(xtime_varying_known_reals)<br>
print(xtime_varying_unknown_reals)<br>
print(2,Train)<br>
training = TimeSeriesDataSet(<br>
data=Train,<br>
add_encoder_length=True,<br>
add_relative_time_idx=True,<br>
add_target_scales=True,<br>
allow_missing_timesteps=True,<br>
group_ids=[‘Group’],<br>
max_encoder_length=10,<br>
max_prediction_length=20,<br>
min_encoder_length=5,<br>
min_prediction_length=1,</p>
<h1><a name="time_idxunique-3" class="anchor" href="#time_idxunique-3"></a>time_idx=‘unique’,</h1>
<pre><code>time_idx='time_idx',
static_categoricals=[], # Categorical features that do not change over time - list
static_reals=[], # Continuous features that do not change over time - list
time_varying_known_categoricals=xtime_varying_known_categoricals, # Known in the future - list
time_varying_unknown_categoricals=[],
target=xtarget,
target_normalizer=xtarget_normalizer,
time_varying_known_reals=xtime_varying_known_reals,
time_varying_unknown_reals=xtime_varying_unknown_reals
</code></pre>
<p>)<br>
validation = TimeSeriesDataSet.from_dataset(training, Train, predict=False, stop_randomization=True)<br>
print(“Passed validation”)<br>
train_dataloader = training.to_dataloader(train=True, batch_size=16, num_workers=32,shuffle=False)<br>
print(“Created dataloaders - show first batch”)<br>
warnings.filterwarnings(“ignore”)<br>
print(“Start train_dataloader comparison with baseline”)<br>
device = torch.device(“cpu”)<br>
print(“actuals calculated”)<br>
Baseline().to(device)<br>
baseline_predictions = Baseline().predict(train_dataloader)<br>
baseline_predictions = torch.cat([b.clone().detach() for b in baseline_predictions])<br>
print(“baseline_predictions calculated”)<br>
val_loss=1e6<br>
early_stop_callback = EarlyStopping(monitor=“val_loss”, min_delta=1e-4, patience=10, verbose=False, mode=“min”)<br>
lr_logger = LearningRateMonitor()  # log the learning rate<br>
logger = TensorBoardLogger(“lightning_logs”)  # logging results to a tensorboard<br>
trainer = Trainer(<br>
callbacks=[lr_logger, early_stop_callback],<br>
enable_model_summary=True,<br>
gradient_clip_val=0.1,<br>
limit_train_batches=10,  # comment in for training, running valiation every 30 batches<br>
logger=logger<br>
)<br>
model = TFT(<br>
training,<br>
attention_head_size=1,<br>
dropout=0.1,<br>
hidden_continuous_size=8,<br>
learning_rate=0.03, # 1<br>
log_interval=10,<br>
lstm_layers=1, # could be interesting<br>
output_size=1, # number of predictors<br>
reduce_on_plateau_patience=4,<br>
)<br>
print(f"\nNumber of parameters in network: {model.size()/1e3:.1f}k\n")</p>

ptrblck: <p>I can reproduce the issue by explicitly passing <code>None</code> to <code>dtype</code> and <code>device</code>:</p>
<pre><code class="lang-python">a = tuple((2, 3))
torch.empty(a, dtype=None, device=None)
# TypeError: empty() received an invalid combination of arguments - got (tuple, device=NoneType, dtype=NoneType), but expected one of:
#  * (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
#  * (tuple of ints size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
</code></pre>
<p>Your code is unfortunately not properly formatted and not executable so I’m unsure if this is the same root cause causing your code to fail.</p>

Loggy: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> for looking.</p>
<p>I’d been staring at this for some time now wondering whether it was one or more of the arguments, were they correctly specified etc, was there a bug that I’d tripped over, should I move the problem to tensorflow … I even posted it as an issue on github as I hadn’t found this list!</p>
<p>But the solution was there all the time staring me in the face!</p>
<p>The model instantiation should be:</p>
<p>model = TFT.from_dataset( … }.</p>
<p>I forgot the ‘.from_dataset’.  I’m new to this game so bear with me. <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"></p>