RuntimeError: For unbatched 2-D input, hx should also be 2-D but got 3-D tensor

Arupreza: <pre><code class="lang-auto">class GRU_Model(nn.Module):
    def __init__(self, num_inputs, num_hidden, num_layers, num_outputs):
        super(GRU_Model, self).__init__()
        self.D = num_inputs
        self.M = num_hidden
        self.K = num_outputs
        self.L = num_layers
        
        self.gru = nn.GRU(
            input_size=self.D,
            hidden_size=self.M,
            num_layers=self.L,
            batch_first=True)
        
        self.fc = nn.Linear(self.M, self.K)
        
    def forward(self, X):
        h0 = torch.zeros(self.L, X.size(0), self.M).to(device) # fix hidden state size
        out, _ = self.gru(X, h0)
        
        out = self.fc(out[:, -1, :])
        return out
</code></pre>
<pre><code class="lang-auto"># get some random training images
dataiter = iter(train_loader)
sets, labels = dataiter.next()

print(sets[0].shape)
print(sets[0].ndim)

print(sets.shape)
print(sets.ndim)

torch.Size([100, 2, 1])
3
torch.Size([32, 100, 2, 1])
4
</code></pre>
<pre><code class="lang-auto">from torchviz import make_dot

netD = model.to(device) # move the model to the same device as the input tensor
x = torch.zeros(100, 2, 1).to(device) # create a dummy input tensor on the same device
out = netD(x)
print("out shape:", out.shape)
make_dot(out, params=dict(netD.named_parameters()))

![Capture|525x500](upload://3HdgozuIQJTTWKvnbLIKfLYYgLL.png)

![Capture|690x112](upload://vyXntHcf5QEtLXHDzFjuDeJ7srx.png)

</code></pre>
<p>Please anyone help me out from this, thanks in advance.</p>

ptrblck: <p>I’m unsure why the error is raised as it seems you are passing a batched 3D input to the model. Are you seeing this issue using code code snippet or at another line of code?</p>

Arupreza: <p>I make the dataset this way. But other code here I upload as it is.</p>
<pre><code class="lang-auto">model = GRU_Model(num_inputs=1, num_hidden=5, num_layers=1, num_outputs=4)
model.to(device)
</code></pre>
<pre><code class="lang-auto">def data_pep(df, feat_col_idx, target_col_idx, seq_len=100):
    X_out = []
    y_out = []
    X = df[:, feat_col_idx]
    y = df[:, target_col_idx]

    start_idx = 0
    end_idx = seq_len
    length = len(df)

    while end_idx &lt;= length:
        X_seq = X[start_idx:end_idx].reshape(seq_len, -1, 1)
        y_seq = y[end_idx-1].reshape(-1, 1)
        X_out.append(X_seq)
        y_out.append(y_seq)
        start_idx += seq_len
        end_idx += seq_len

    X_out = np.array(X_out)
    y_out = np.array(y_out)
    
    return X_out, y_out
</code></pre>
<pre><code class="lang-auto">from torch.utils.data import DataLoader, Dataset

class CustomDataset(Dataset):
    def __init__(self, X, y):
        self.X = X
        self.y = y
        
    def __len__(self):
        return len(self.X)
    
    def __getitem__(self, idx):
        X_item = self.X[idx]
        y_item = self.y[idx]
        return X_item, y_item
</code></pre>
<pre><code class="lang-auto">def train_one_epoch(model, train_loader, optimizer, device, criterion):
    """Train model for one epoch and return the mean train_loss."""
    model.train()
    running_loss_train = 0
    for inputs, labels in train_loader:
        labels = labels.type(torch.LongTensor)
        inputs = inputs.type(torch.cuda.FloatTensor)
        #labels = labels.type(torch.cuda.FloatTensor)
        inputs, labels = inputs.to(device=device), labels.to(device=device)
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        running_loss_train += loss.item()
    train_loss = running_loss_train / len(train_loader.dataset)
    return train_loss
</code></pre>
<pre><code class="lang-auto">def validate(model, valid_loader, device, criterion):
    """Validate model and return the accuracy and mean loss."""
    model.eval()
    correct = 0
    running_loss_val = 0
    with torch.no_grad():
        for inputs, labels in valid_loader:
            labels = labels.type(torch.LongTensor)
            inputs = inputs.type(torch.cuda.FloatTensor)
            #labels = labels.type(torch.cuda.FloatTensor)
            inputs, labels = inputs.to(device=device), labels.to(device=device)
            outputs = model(inputs)
            loss = criterion(outputs, labels)
            pred = outputs.argmax(dim=1)
            correct += pred.eq(labels).sum().item()
            running_loss_val += loss.item()
    val_acc = correct / len(valid_loader.dataset)
    val_loss = running_loss_val / len(valid_loader.dataset)
    return val_acc, val_loss
</code></pre>
<pre><code class="lang-auto">def fit(model, train_loader, valid_loader, learning_rate, num_epochs):
    criterion = nn.CrossEntropyLoss(reduction='sum')
    optimizer = optim.Adam(model.parameters(), lr=learning_rate)
    es = EarlyStopping(mode='min', path='./x.pth', patience=10)
    model = model.to(device)
    scheduler = ExponentialLR(optimizer, gamma=0.1)

    for epoch in range(1, num_epochs + 1):
        train_loss = train_one_epoch(model, train_loader, optimizer, device, criterion)
        val_acc, val_loss = validate(model, valid_loader, device, criterion)
        scheduler.step()
        print(f'Epoch {epoch:2}/{num_epochs}',
              f'train loss: {train_loss:.4f}',
              f'val loss: {val_loss:.4f}',
              f'val acc: {val_acc:.2%}',
              sep=' | '
             )
        if es(val_loss, model):
            break
</code></pre>

ptrblck: <p>Your initial code works fine for me:</p>
<pre><code class="lang-python">device = "cpu"
model = GRU_Model(1, 1, 1, 1)

netD = model.to(device) # move the model to the same device as the input tensor
x = torch.zeros(100, 2, 1).to(device) # create a dummy input tensor on the same device
out = netD(x)
print("out shape:", out.shape)
# out shape: torch.Size([100, 1])
make_dot(out, params=dict(netD.named_parameters()))
</code></pre>
<p>and returns a valid output so I would need more information how to reproduce the issue and where the error is raised from.</p>

Arupreza: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Actually I tried to find the problem but I could not find out. For that reason I restarted to build the model now it worked. Thanks for your help.</p>

Fede: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
import pandas as pd<br>
import torch<br>
import torch.nn as nn<br>
import torch.optim as optim<br>
from torch.utils.data import DataLoader, Dataset<br>
from sklearn.preprocessing import LabelEncoder<br>
from sklearn.model_selection import train_test_split</p>
<h1><a name="carica-i-dati-1" class="anchor" href="#carica-i-dati-1"></a>Carica i dati</h1>
<p>data = pd.read_csv(r"C:/Users/crucillf/OneDrive - STMicroelectronics/Documents/CSV/329/dsrv.rule.diags.aut.2024-02-01_03_55_56.506428.csv")</p>
<h1><a name="stampa-i-nomi-delle-colonne-per-verificare-2" class="anchor" href="#stampa-i-nomi-delle-colonne-per-verificare-2"></a>Stampa i nomi delle colonne per verificare</h1>
<p>print(“Colonne del file CSV:”, data.columns)</p>
<h1><a name="la-colonna-da-visualizzare-p-3" class="anchor" href="#la-colonna-da-visualizzare-p-3"></a>La colonna da visualizzare è ‘P’</h1>
<p>column_to_display = ‘P’</p>
<h1><a name="assicurati-che-la-colonna-esista-nel-dataframe-4" class="anchor" href="#assicurati-che-la-colonna-esista-nel-dataframe-4"></a>Assicurati che la colonna esista nel DataFrame</h1>
<p>if column_to_display in data.columns:<br>
# Visualizza la colonna<br>
print(data[column_to_display])<br>
else:<br>
print(f"Errore: ‘{column_to_display}’ non esiste nel DataFrame.")<br>
exit()</p>
<h1><a name="prepara-i-dati-per-laddestramento-5" class="anchor" href="#prepara-i-dati-per-laddestramento-5"></a>Prepara i dati per l’addestramento</h1>
<p>class LogDataset(Dataset):<br>
def <strong>init</strong>(self, logs, labels):<br>
self.logs = logs<br>
self.labels = labels</p>
<pre><code>def __len__(self):
    return len(self.logs)

def __getitem__(self, idx):
    log = self.logs[idx]
    label = self.labels[idx]
    return log, label
</code></pre>
<p>logs = data[‘P’].astype(str).tolist()  # Converti tutti i valori in stringhe<br>
labels = data[‘P’].astype(str).tolist()  # Converti tutti i valori in stringhe</p>
<h1><a name="codifica-le-etichette-6" class="anchor" href="#codifica-le-etichette-6"></a>Codifica le etichette</h1>
<p>label_encoder = LabelEncoder()<br>
labels = label_encoder.fit_transform(labels)</p>
<h1><a name="dividi-i-dati-in-training-e-test-set-7" class="anchor" href="#dividi-i-dati-in-training-e-test-set-7"></a>Dividi i dati in training e test set</h1>
<p>logs_train, logs_test, labels_train, labels_test = train_test_split(logs, labels, test_size=0.2, random_state=42)</p>
<h1><a name="trova-la-lunghezza-massima-delle-sequenze-8" class="anchor" href="#trova-la-lunghezza-massima-delle-sequenze-8"></a>Trova la lunghezza massima delle sequenze</h1>
<p>max_length = max(len(log) for log in logs)</p>
<h1><a name="funzione-per-pad-le-sequenze-9" class="anchor" href="#funzione-per-pad-le-sequenze-9"></a>Funzione per pad le sequenze</h1>
<p>def pad_sequence(seq, max_length):<br>
return seq + ’ ’ * (max_length - len(seq))</p>
<h1><a name="pad-le-sequenze-nei-dataset-10" class="anchor" href="#pad-le-sequenze-nei-dataset-10"></a>Pad le sequenze nei dataset</h1>
<p>logs_train = [pad_sequence(log, max_length) for log in logs_train]<br>
logs_test = [pad_sequence(log, max_length) for log in logs_test]</p>
<h1><a name="crea-un-set-di-tutti-i-caratteri-unici-11" class="anchor" href="#crea-un-set-di-tutti-i-caratteri-unici-11"></a>Crea un set di tutti i caratteri unici</h1>
<p>unique_chars = set(‘’.join(logs_train + logs_test))<br>
char_to_index = {char: idx for idx, char in enumerate(unique_chars)}</p>
<h1><a name="crea-i-dataset-e-i-dataloader-12" class="anchor" href="#crea-i-dataset-e-i-dataloader-12"></a>Crea i dataset e i dataloader</h1>
<p>train_dataset = LogDataset(logs_train, labels_train)<br>
test_dataset = LogDataset(logs_test, labels_test)  # Correzione: aggiunto labels_test<br>
train_dataloader = DataLoader(train_dataset, batch_size=64, shuffle=True)<br>
test_dataloader = DataLoader(test_dataset, batch_size=64, shuffle=False)</p>
<h1><a name="definisci-il-modello-rnn-13" class="anchor" href="#definisci-il-modello-rnn-13"></a>Definisci il modello RNN</h1>
<p>class RNNModel(nn.Module):<br>
def <strong>init</strong>(self, input_size, hidden_size, output_size, num_layers=1):<br>
super(RNNModel, self).<strong>init</strong>()<br>
self.hidden_size = hidden_size<br>
self.num_layers = num_layers<br>
self.embedding = nn.Embedding(input_size, hidden_size)<br>
self.lstm = nn.LSTM(hidden_size, hidden_size, num_layers, batch_first=True)<br>
self.fc = nn.Linear(hidden_size, output_size)</p>
<pre><code>def forward(self, x):
    h_0 = Variable(torch.zeros(self.num_layers_LSTM, x.size(0), self.hidden_dim_LSTM)).requires_grad_().to(device) #hidden state
    c_0 = Variable(torch.zeros(self.num_layers_LSTM, x.size(0), self.hidden_dim_LSTM)).requires_grad_().to(device) #internal state

    x = self.embedding(x)
    h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(x.device)
    c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(x.device)
    out, _ = self.lstm(x, (h0, c0))
    out = out[:, -1, :]
    out = self.fc(out)
    return out
</code></pre>
<h1><a name="parametri-del-modello-14" class="anchor" href="#parametri-del-modello-14"></a>Parametri del modello</h1>
<p>input_size = len(unique_chars)  # Numero di caratteri unici<br>
hidden_size = 64<br>
output_size = len(set(labels))<br>
num_layers = 1</p>
<h1><a name="inizializza-il-modello-la-loss-function-e-loptimizer-15" class="anchor" href="#inizializza-il-modello-la-loss-function-e-loptimizer-15"></a>Inizializza il modello, la loss function e l’optimizer</h1>
<p>device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)<br>
model = RNNModel(input_size, hidden_size, output_size, num_layers).to(device)<br>
criterion = nn.CrossEntropyLoss()<br>
optimizer = optim.AdamW(model.parameters(), lr=0.001)</p>
<h1><a name="addestramento-del-modello-16" class="anchor" href="#addestramento-del-modello-16"></a>Addestramento del modello</h1>
<p>model.train()<br>
num_epochs = 3  # Numero di epoche<br>
for epoch in range(num_epochs):<br>
epoch_loss = 0<br>
for logs, labels in train_dataloader:<br>
logs = torch.tensor([[char_to_index[char] for char in log] for log in logs], dtype=torch.long).to(device)<br>
labels = torch.tensor(labels, dtype=torch.long).clone().detach().to(device)</p>
<pre><code>    optimizer.zero_grad()
    outputs = model(logs)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()
    
    epoch_loss += loss.item()

avg_loss = epoch_loss / len(train_dataloader)
print(f'Epoca [{epoch+1}/{num_epochs}], Perdita: {avg_loss:.4f}')
</code></pre>
<h1><a name="valutazione-del-modello-17" class="anchor" href="#valutazione-del-modello-17"></a>Valutazione del modello</h1>
<p>model.eval()<br>
correct = 0<br>
total = 0<br>
with torch.no_grad():<br>
for logs, labels in test_dataloader:<br>
logs = torch.tensor([[char_to_index[char] for char in log] for log in logs], dtype=torch.long).to(device)<br>
labels = torch.tensor(labels, dtype=torch.long).clone().detach().to(device)</p>
<pre><code>    outputs = model(logs)
    _, predicted = torch.max(outputs.data, 1)
    total += labels.size(0)
    correct += (predicted == labels).sum().item()
</code></pre>
<p>print(f’Accuracy: {100 * correct / total}%')</p>
<h1><a name="filtra-i-dati-a-partire-dal-12-gennaio-2024-e-con-soglia-minima-di-10-18" class="anchor" href="#filtra-i-dati-a-partire-dal-12-gennaio-2024-e-con-soglia-minima-di-10-18"></a>Filtra i dati a partire dal 12 gennaio 2024 e con soglia minima di 1.0</h1>
<p>data[‘P’] = pd.to_numeric(data[‘P’], errors=‘coerce’)<br>
data[‘Date’] = pd.to_datetime(data[‘P’], errors=‘coerce’)  # Usa la colonna ‘P’ come data<br>
start_date = pd.to_datetime(‘2024-01-12’)<br>
filtered_data = data[(data[‘Date’] &gt;= start_date) &amp; (data[‘P’] &gt; 1.0)]</p>
<h1><a name="crea-un-nuovo-dataframe-con-i-dati-filtrati-19" class="anchor" href="#crea-un-nuovo-dataframe-con-i-dati-filtrati-19"></a>Crea un nuovo DataFrame con i dati filtrati</h1>
<p>filtered_data = filtered_data[[‘P’]]</p>
<h1><a name="prepara-i-nuovi-log-per-la-predizione-20" class="anchor" href="#prepara-i-nuovi-log-per-la-predizione-20"></a>Prepara i nuovi log per la predizione</h1>
<p>new_logs = filtered_data[‘P’].astype(str).tolist()<br>
new_logs = [pad_sequence(log, max_length) for log in new_logs]</p>
<h1><a name="gestione-dei-caratteri-sconosciuti-21" class="anchor" href="#gestione-dei-caratteri-sconosciuti-21"></a>Gestione dei caratteri sconosciuti</h1>
<p>def char_to_index_safe(char):<br>
return char_to_index.get(char, char_to_index[’ '])  # Sostituisci con spazio se il carattere non è trovato</p>
<p>new_logs_tensor = torch.tensor([[char_to_index_safe(char) for char in log] for log in new_logs], dtype=torch.long).to(device)</p>
<h1><a name="aggiungi-una-dimensione-batch-se-necessario-22" class="anchor" href="#aggiungi-una-dimensione-batch-se-necessario-22"></a>Aggiungi una dimensione batch se necessario</h1>
<p>if new_logs_tensor.dim() == 2:<br>
new_logs_tensor = new_logs_tensor.unsqueeze(1)  # Aggiungi una dimensione batch</p>
<p>outputs = model(new_logs_tensor)<br>
predictions = torch.argmax(outputs, dim=1)<br>
print(predictions)<br>
this is my code and I have the same error how to fix it?</p>

ptrblck: <p>Your code is unfortunately not properly formatted and also depends on unknown data.<br>
If you get stuck in debugging this issue, please post a minimal and executable code snippet reproducing the issue.</p>

Fede: <p><a href="https://pastebin.com/hiYUrQvX" class="inline-onebox" rel="noopener nofollow ugc">import pandas as pdimport torchimport torch.nn as nnimport torch.optim as - Pastebin.com</a> here’s my code for you</p>

ptrblck: <p>You are still using an unknown data dependency.</p>
<pre data-code-wrap="python"><code class="lang-python"># Carica i dati
data = pd.read_csv(r"C:/Users/crucillf/OneDrive - STMicroelectronics/Documents/CSV/329/dsrv.rule.diags.aut.2024-02-01_03_55_56.506428.csv")
</code></pre>

Fede: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> No, I’m not using an unknown data dependency, I have it in my PC</p>

ptrblck: <p>Unfortunately this won’t help us in reproducing and debgging the issue, since we cannot access your PC.</p>

Fede: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> the error this is<br>
Exception has occurred: RuntimeError For unbatched 2-D input, hx and cx should also be 2-D but got (3-D, 3-D) tensors File “C:\Users\crucillf\OneDrive - STMicroelectronics\Documents\Tesi\Script Python\Example_of_RNN3.py”, line 85, in forward out, _ = self.lstm(x, (h0, c0)) ^^^^^^^^^^^^^^^^^^^^^^ File “C:\Users\crucillf\OneDrive - STMicroelectronics\Documents\Tesi\Script Python\Example_of_RNN3.py”, line 161, in  outputs = model(new_logs_tensor) ^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: For unbatched 2-D input, hx and cx should also be 2-D but got (3-D, 3-D) tensors<br>
can you help me?</p>

Fede: <p>i send you my error can you help me?</p>