LSTMSeq2seq for prediction series time

username: CARLOS7
cooked: <p>I am trying to make a neural network type LSTMseq2seq, where with 11 variables that I have I want to predict only 1, but I think that something in the structure is wrong, since it gives me negative r2 values</p>

username: CARLOS7
cooked: <p>first_index = 746</p>
<p>last_index = 1910</p>
<h1><a name="selecciona-los-datos-del-pozo-1" class="anchor" href="#selecciona-los-datos-del-pozo-1"></a>Selecciona los datos del pozo</h1>
<p>df_well = model.loc[first_index:last_index]</p>
<p>x = df_well.drop([‘NPD_WELL_BORE_CODE’,‘AVG_DOWNHOLE_PRESSURE’,‘DATEPRD’], axis=1)</p>
<p>y = df_well[[‘AVG_DOWNHOLE_PRESSURE’]]</p>
<p>train_size = int(0.90 * len(df_well))</p>
<p>x_train, y_train = x[:train_size], y[:train_size]</p>
<p>x_test, y_test = x[train_size:], y[train_size:]</p>
<p>import torch<br>
import torch.nn as nn<br>
import torch.optim as optim<br>
from torch.utils.data import DataLoader, TensorDataset</p>
<p>class Seq2SeqModel(nn.Module):<br>
def <strong>init</strong>(self, input_size, hidden_size, output_size, window_size, horizon):<br>
super(Seq2SeqModel, self).<strong>init</strong>()<br>
self.encoder = nn.LSTM(input_size, hidden_size,num_layers=11 )<br>
self.decoder = nn.LSTM(hidden_size, hidden_size,num_layers=11)<br>
self.linear = nn.Linear(hidden_size, output_size)</p>
<pre><code>def forward(self, x):
    # Encoder
    _, (hidden, cell) = self.encoder(x)

    # Decoder
    decoder_input = torch.zeros_like(x)  # Inicializar con ceros
    output, _ = self.decoder(decoder_input, (hidden, cell))

    # Salida
    output = self.linear(output.squeeze(0))
    return output
</code></pre>
<h1><a name="preparar-datos-2" class="anchor" href="#preparar-datos-2"></a>Preparar datos</h1>
<p>x_train_tensor = torch.tensor(x_train.values, dtype=torch.float32)<br>
y_train_tensor = torch.tensor(y_train.values, dtype=torch.float32)</p>
<p>x_test_tensor = torch.tensor(x_test.values, dtype=torch.float32)<br>
y_test_tensor = torch.tensor(y_test.values, dtype=torch.float32)</p>
<h1><a name="crear-conjuntos-de-datos-y-data-loaders-3" class="anchor" href="#crear-conjuntos-de-datos-y-data-loaders-3"></a>Crear conjuntos de datos y data loaders</h1>
<p>train_dataset = TensorDataset(x_train_tensor, y_train_tensor)<br>
test_dataset = TensorDataset(x_test_tensor, y_test_tensor)</p>
<p>batch_size = 40<br>
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)<br>
test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)</p>
<h1><a name="configurar-modelo-funcin-de-prdida-y-optimizador-4" class="anchor" href="#configurar-modelo-funcin-de-prdida-y-optimizador-4"></a>Configurar modelo, función de pérdida y optimizador</h1>
<p><span class="hashtag-raw">#input_size</span> = x_train.shape[1]<br>
<span class="hashtag-raw">#output_size</span> = y_train.shape[1]<br>
<span class="hashtag-raw">#hidden_size</span> = 11</p>
<p><span class="hashtag-raw">#model</span> = Seq2SeqModel(input_size, hidden_size, output_size)  # Reemplaza ‘TuModeloLSTM’ con el nombre de tu modelo<br>
<span class="hashtag-raw">#criterion</span> = nn.MSELoss()  # Puedes cambiar la función de pérdida según tus necesidades<br>
<span class="hashtag-raw">#optimizer</span> = torch.optim.Adam(model.parameters(), lr=0.4)  # Puedes ajustar el optimizador y la tasa de aprendizaje</p>
<h1><a name="configurar-modelo-funcin-de-prdida-y-optimizador-5" class="anchor" href="#configurar-modelo-funcin-de-prdida-y-optimizador-5"></a>Configurar modelo, función de pérdida y optimizador</h1>
<p>input_size = x_train.shape[1]<br>
output_size = y_train.shape[1]<br>
hidden_size = 11<br>
window_size = 40  # Ajusta según sea necesario<br>
horizon = 40  # Ajusta según sea necesario</p>
<p>model = Seq2SeqModel(input_size, hidden_size, output_size, window_size, horizon)<br>
criterion = nn.MSELoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)</p>
<h1><a name="entrenamiento-6" class="anchor" href="#entrenamiento-6"></a>Entrenamiento</h1>
<p>num_epochs=200<br>
for epoch in range(num_epochs):<br>
for batch_x, batch_y in train_loader:<br>
optimizer.zero_grad()</p>
<pre><code>    if isinstance(batch_x, torch.Tensor) and batch_x.is_sparse:
        batch_x = batch_x.to_dense()
    output = model(batch_x)
    loss = criterion(output, batch_y)
    loss.backward()
    optimizer.step()

# Imprimir la pérdida al final de cada época
print(f'Época {epoch + 1}/{num_epochs}, Pérdida: {loss.item()}')  
</code></pre>
<p>Época 143/200, Pérdida: 356.6650390625 Época 144/200, Pérdida: 120.84323120117188 Época 145/200, Pérdida: 407.1255798339844 Época 146/200, Pérdida: 966.4486083984375 Época 147/200, Pérdida: 778.9110107421875 Época 148/200, Pérdida: 379.4190368652344 Época 149/200, Pérdida: 201.9375762939453 Época 150/200, Pérdida: 998.202880859375 Época 151/200, Pérdida: 7103.08642578125 Época 152/200, Pérdida: 583.3623046875 Época 153/200, Pérdida: 581.837646484375 Época 154/200, Pérdida: 197.0320281982422 Época 155/200, Pérdida: 6781.46630859375 Época 156/200, Pérdida: 149.66714477539062 Época 157/200, Pérdida: 1184.738037109375 Época 158/200, Pérdida: 1178.5792236328125 Época 159/200, Pérdida: 150.41824340820312 Época 160/200, Pérdida: 562.591552734375 Época 161/200, Pérdida: 285.3613586425781 Época 162/200, Pérdida: 273.0677490234375 Época 163/200, Pérdida: 1095.458251953125 Época 164/200, Pérdida: 6904.693359375 Época 165/200, Pérdida: 269.6636047363281 Época 166/200, Pérdida: 399.09637451171875 Época 167/200, Pérdida: 87.79035186767578 Época 168/200, Pérdida: 6977.03955078125 Época 169/200, Pérdida: 881.690673828125 Época 170/200, Pérdida: 299.3782653808594 Época 171/200, Pérdida: 597.3984375 Época 172/200, Pérdida: 854.1024780273438 Época 173/200, Pérdida: 1044.854736328125 Época 174/200, Pérdida: 99.43424987792969 Época 175/200, Pérdida: 857.2242431640625 Época 176/200, Pérdida: 1315.69384765625 Época 177/200, Pérdida: 138.40518188476562 Época 178/200, Pérdida: 373.2205810546875 Época 179/200, Pérdida: 1005.6658325195312 Época 180/200, Pérdida: 336.55047607421875 Época 181/200, Pérdida: 225.5845184326172 Época 182/200, Pérdida: 13401.2568359375 Época 183/200, Pérdida: 194.87428283691406 Época 184/200, Pérdida: 187.71463012695312 Época 185/200, Pérdida: 6852.80712890625 Época 186/200, Pérdida: 1094.6590576171875 Época 187/200, Pérdida: 216.69117736816406 Época 188/200, Pérdida: 13514.875 Época 189/200, Pérdida: 741.24853515625 Época 190/200, Pérdida: 268.2852783203125 Época 191/200, Pérdida: 843.3907470703125 Época 192/200, Pérdida: 243.44464111328125 Época 193/200, Pérdida: 745.4027099609375 Época 194/200, Pérdida: 6966.73779296875 Época 195/200, Pérdida: 7471.0400390625 Época 196/200, Pérdida: 345.9413757324219 Época 197/200, Pérdida: 228.78480529785156 Época 198/200, Pérdida: 1013.11279296875 Época 199/200, Pérdida: 152.55259704589844 Época 200/200, Pérdida: 791.8948364257812</p>
<p>from sklearn.metrics import r2_score, mean_absolute_error<br>
import numpy as np</p>
<h1><a name="poner-el-modelo-en-modo-de-evaluacin-7" class="anchor" href="#poner-el-modelo-en-modo-de-evaluacin-7"></a>Poner el modelo en modo de evaluación</h1>
<p>model.eval()</p>
<h1><a name="listas-para-almacenar-las-predicciones-y-etiquetas-reales-8" class="anchor" href="#listas-para-almacenar-las-predicciones-y-etiquetas-reales-8"></a>Listas para almacenar las predicciones y etiquetas reales</h1>
<p>predictions = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
true_labels = <span class="chcklst-box fa fa-square-o fa-fw"></span></p>
<h1><a name="realizar-predicciones-en-el-conjunto-de-prueba-9" class="anchor" href="#realizar-predicciones-en-el-conjunto-de-prueba-9"></a>Realizar predicciones en el conjunto de prueba</h1>
<p>with torch.no_grad():<br>
for batch_x, batch_y in test_loader:<br>
# Asegurarse de que batch_x sea denso<br>
if isinstance(batch_x, torch.Tensor) and batch_x.is_sparse:<br>
batch_x = batch_x.to_dense()</p>
<pre><code>    # Obtener predicciones del modelo
    output = model(batch_x)
    
    # Almacenar predicciones y etiquetas reales
    predictions.append(output.numpy())
    true_labels.append(batch_y.numpy())
</code></pre>
<h1><a name="concatenar-y-convertir-las-listas-en-tensores-10" class="anchor" href="#concatenar-y-convertir-las-listas-en-tensores-10"></a>Concatenar y convertir las listas en tensores</h1>
<p>predictions = np.concatenate(predictions, axis=0)<br>
true_labels = np.concatenate(true_labels, axis=0)</p>
<h1><a name="calcular-la-mtrica-de-evaluacin-por-ejemplo-mse-11" class="anchor" href="#calcular-la-mtrica-de-evaluacin-por-ejemplo-mse-11"></a>Calcular la métrica de evaluación (por ejemplo, MSE)</h1>
<p>mse = ((predictions - true_labels) ** 2).mean()</p>
<p>print(f’MSE en el conjunto de prueba: {mse}')</p>
<h1><a name="calcular-r2-12" class="anchor" href="#calcular-r2-12"></a>Calcular R^2</h1>
<p>r2 = r2_score(true_labels, predictions)<br>
print(f’R^2 en el conjunto de prueba: {r2}')</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/7/67757e3720664c6ccf637c78caed197739927511.png" data-download-href="https://discuss.pytorch.org/uploads/default/67757e3720664c6ccf637c78caed197739927511" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67757e3720664c6ccf637c78caed197739927511_2_690x443.png" alt="image" data-base62-sha1="eLeUeVwudP35Ps6mlX9j5gJq29r" width="690" height="443" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/7/67757e3720664c6ccf637c78caed197739927511_2_690x443.png, https://discuss.pytorch.org/uploads/default/original/3X/6/7/67757e3720664c6ccf637c78caed197739927511.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/7/67757e3720664c6ccf637c78caed197739927511.png 2x" data-dominant-color="F8F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">852×548 22.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: CARLOS7
cooked: <p>Someone who can help me fix my network, I would be very grateful and I would give you a small reward.</p>

username: vdw
cooked: <p>Please fix your formatting of your code; it’s  very difficult read.</p>
<p>Apart from that, you encoder-decoder setup looks odd:</p>
<ul>
<li>
<p>If your input sequence is of the length <code>L</code>, you decoder will return a sequence of length <code>L</code></p>
</li>
<li>
<p>All the inputs of you decoder (i.e., the input at any time step is a zero vector), but the input of the (i+1)-th step should depend on the output of the i-th step. At least intuitively; I’m not 100% sure what your intended learning task is.</p>
</li>
</ul>

username: CARLOS7
cooked: <p>The task I want to do is that with my 11 known variables I want to predict only one 1 variable, which in this case is in y</p>

username: J_Johnson
cooked: <p>LSTM expects batch on dim=1, unless you specify <code>batch_first=True</code> when instantiating the layer.</p>
<p>What are the dims and sizes going into your model? For example: [batch_size, sequence_length, features]</p>

username: vdw
cooked: <p>Predicting 1 variable once or 1 variable for multiple time steps. For example, given some weather data, do you want to predict just the temperature for the next day or for the next k days.</p>
<p>If it’s the former, you wouldn’t need a encoder-decoder architecture.</p>