TimeSeriesTransformerModel - 'features' Is 'NoneType'
Author: LtlSh
Pull request: no

### System Info





Sat Mar  4 10:44:51 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 527.41       Driver Version: 527.41       CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ... WDDM  | 00000000:01:00.0  On |                  N/A |
| 30%   37C    P8    18W / 350W |    550MiB / 24576MiB |     27%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1580    C+G   ...y\ShellExperienceHost.exe    N/A      |
|    0   N/A  N/A      5332    C+G   ...e\PhoneExperienceHost.exe    N/A      |
|    0   N/A  N/A      7560    C+G   ...logioptionsplus_agent.exe    N/A      |
|    0   N/A  N/A      7672    C+G   ...5n1h2txyewy\SearchApp.exe    N/A      |
|    0   N/A  N/A      8268    C+G   C:\Windows\explorer.exe         N/A      |
|    0   N/A  N/A     11324    C+G   ...cw5n1h2txyewy\LockApp.exe    N/A      |
|    0   N/A  N/A     12316    C+G   ...(x86)\AnyDesk\AnyDesk.exe    N/A      |
|    0   N/A  N/A     12908    C+G   ...ge\Application\msedge.exe    N/A      |
|    0   N/A  N/A     13064    C+G   ...2txyewy\TextInputHost.exe    N/A      |
|    0   N/A  N/A     13764    C+G   ...3d8bbwe\CalculatorApp.exe    N/A      |
|    0   N/A  N/A     14880    C+G   ...lPanel\SystemSettings.exe    N/A      |
+-----------------------------------------------------------------------------+



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Hi, 

We're trying to train TimeSeriesTransformerModel, but it seems that there's a variable called 'features' that's not getting assigned. However, we can't figure out why is this happening.

Our code:

`
# Initializing a default Time Series Transformer configuration
configuration = TimeSeriesTransformerConfig(prediction_length = 327, lags_sequence = [0, 0, 0])

# Randomly initializing a model (with random weights) from the configuration
model = TimeSeriesTransformerForPrediction(configuration)

# Accessing the model configuration
configuration = model.config
#we dont know if passing the data as a dataframe instead if a tesndor would work
#currently model.train() is throwing an error, maybe we need to use a gpu? TODO
# Setting the model to training mode
model.train()

# Defining the loss function and optimizer
loss_fn = torch.nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

# Training loop
for epoch in range(100):
    for batch in dataloader:
        # Forward pass
        outputs =  model(
        past_values=batch["past_values"],
        past_time_features=batch["past_time_features"],
        past_observed_mask=None,
        static_categorical_features=None,
        static_real_features=None,
        future_values=batch["future_values"],
        future_time_features=batch["future_time_features"],
                         )
        loss = loss_fn(outputs, batch)

        # Backward pass and optimization
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

    # Printing the training loss
    if (epoch + 1) % 10 == 0:
        print(f"Epoch [{epoch + 1}/100], Loss: {loss.item()}")
`

We're getting this error:

`Traceback (most recent call last):
  File "D:\Final Project\fMRI_Ariel_Lital\train.py", line 58, in <module>
    outputs =  model(
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\transformers\models\time_series_transformer\modeling_time_series_transformer.py", line 1813, in forward
    outputs = self.model(
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\transformers\models\time_series_transformer\modeling_time_series_transformer.py", line 1626, in forward
    transformer_inputs, scale, static_feat = self.create_network_inputs(
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\transformers\models\time_series_transformer\modeling_time_series_transformer.py", line 1532, in create_network_inputs
    embedded_cat = self.embedder(static_categorical_features)
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\transformers\models\time_series_transformer\modeling_time_series_transformer.py", line 250, in forward
    [
  File "C:\Users\Cognition\anaconda3\envs\ArielLital\lib\site-packages\transformers\models\time_series_transformer\modeling_time_series_transformer.py", line 251, in <listcomp>
    embed(cat_feature_slice.squeeze(-1))
AttributeError: 'NoneType' object has no attribute 'squeeze'`


Here is the source code where the error occurs - we noticed 'features' is NoneType but don't know why:
`class FeatureEmbedder(nn.Module):
    def __init__(self, cardinalities: List[int], embedding_dims: List[int]) -> None:
        super().__init__()

        self.num_features = len(cardinalities)
        self.embedders = nn.ModuleList([nn.Embedding(c, d) for c, d in zip(cardinalities, embedding_dims)])

    def forward(self, features: torch.Tensor) -> torch.Tensor:
        if self.num_features > 1:
            # we slice the last dimension, giving an array of length
            # self.num_features with shape (N,T) or (N)
            cat_feature_slices = torch.chunk(features, self.num_features, dim=-1)
        else:
            cat_feature_slices = [**features**]

        return torch.cat(
            [
                embed(cat_feature_slice.squeeze(-1))
                for embed, cat_feature_slice in zip(self.embedders, cat_feature_slices)
            ],
            dim=-1,
        )
`


Would appreciate your help.

@ArthurZucker and @younesbelkadav
[train.txt](https://github.com/huggingface/transformers/files/10887890/train.txt)


### Expected behavior

We would like to train a TimeSeriesTransformerModelfor for forcasting on tabular data.


Comments:
@LtlSh can you kindly try with the main branch of transformers?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.