TFT : AssertionError: filters should not remove entries all entries - check encoder/decoder lengths and lags

himanshu_birla: <p>I am training a temporal fusion transformer and getting AssertionError: filters should not remove entries all entries - check encoder/decoder lengths and lags</p>
<p>My Code :<br>
max_encoder_length: int = train.groupby(‘contract_id’).size().max()</p>
<p>min_encoder_length: int = 1</p>
<p>min_prediction_length: int = 1</p>
<p>max_prediction_length: int = 13</p>
<p>batch_size: int = 32</p>
<p>hidden_size: int = study.best_trial.params[‘hidden_size’]</p>
<p>lstm_layers: int = 3</p>
<p>attention_head_size: int = study.best_trial.params[‘attention_head_size’]</p>
<p>dropout: float = study.best_trial.params[‘dropout’]</p>
<p>hidden_continuous_size: int = study.best_trial.params[‘hidden_continuous_size’]</p>
<p>learning_rate: float = study.best_trial.params[‘learning_rate’]</p>
<p>max_epochs: int = 50</p>
<p>random_seed: int = 0</p>
<blockquote>
<p>Blockquote<br>
training_dataset = TimeSeriesDataSet(<br>
# df_1[lambda x: x.time_idx &lt;= training_cutoff],<br>
train,<br>
time_idx=‘time_idx’,<br>
target=‘NetRevenue’,<br>
group_ids=[‘contract_id’,‘cluster’,‘Sector’],<br>
min_encoder_length=min_encoder_length,<br>
max_encoder_length=max_encoder_length,<br>
min_prediction_length=min_prediction_length,<br>
max_prediction_length=max_prediction_length,<br>
static_categoricals=[‘contract_id’,‘Sector’,‘cluster’],<br>
static_reals=<span class="chcklst-box fa fa-square-o fa-fw"></span>,<br>
time_varying_known_categoricals=[‘<em>flag_zero_revenue</em>’,‘flag_engagement_near_end_indx’],<br>
time_varying_unknown_categoricals=[‘invoice_raised’,‘Amt_Received’],<br>
time_varying_unknown_reals=[<br>
‘NetRevenue’,<br>
‘GLHours’,‘PTOAdjustment’,<br>
‘WorkingDays’],<br>
time_varying_known_reals=[<br>
‘period_sin’,‘period_cos’,‘time_idx’,‘FiscalYear’,‘Period’,‘weighted_days_to_end’],<br>
target_normalizer=GroupNormalizer(<br>
groups=[‘contract_id’,‘cluster’,‘Sector’],transformation=‘softplus’,center=True<br>
),<br>
add_relative_time_idx=True,<br>
add_target_scales=True,</p>
</blockquote>
<pre><code>        add_encoder_length=True,
        allow_missing_timesteps=True,
        categorical_encoders={
            'contract_id': NaNLabelEncoder(add_nan=True, warn=False),
            'cluster': NaNLabelEncoder(add_nan=True, warn=False),
            # 'ProjectTypeDesc': NaNLabelEncoder(add_nan=True, warn=False),
            # 'Sector': NaNLabelEncoder(add_nan=True, warn=False),
            # 'flag_zero_revenue_in_fiscal_year':NaNLabelEncoder(add_nan=True, warn=False),
            # 'NetRevenue': NaNLabelEncoder(add_nan=True, warn=False),
            # 'project_start': NaNLabelEncoder(add_nan=True, warn=False),
            # 'project_end': NaNLabelEncoder(add_nan=True, warn=False),
            # 'fiscal_year': NaNLabelEncoder(add_nan=True, warn=False),
            # 'WorkingDays':  NaNLabelEncoder(add_nan=True, warn=False),  
        }
        
    )
</code></pre>
<p>validation_dataset = TimeSeriesDataSet.from_dataset(<br>
dataset=training_dataset,<br>
data=val,<br>
stop_randomization=True,<br>
predict=True</p>
<pre><code>        # min_prediction_idx = int(training_cutoff + 1)
    )
</code></pre>
<p>I am getting the assertion error while running code for validation_dataset .</p>
<p>If anyone can help me identify, provide solution and explain , will be a great help .</p>