akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
ResUnet implementation in pytorch for medical image segmentation

Saurav_K_Mhptra: <p>I am working on pytorch implementation of resUnet for binary segmentation but my model doesnt learn anything. The validation loss stays constant and  the training loss hardly changes. Here is my resUnet model. can anyone help as to where i am doing wrong.<br>
class batchnorm_relu(nn.Module):<br>
def <strong>init</strong>(self, in_c):<br>
super().<strong>init</strong>()</p>
<pre><code>    self.bn = nn.BatchNorm2d(in_c)
    self.relu = nn.ReLU()

def forward(self, inputs):
    x = self.bn(inputs)
    x = self.relu(x)
    return x
</code></pre>
<p>class residual_block(nn.Module):<br>
def <strong>init</strong>(self, in_c, out_c, stride=1):<br>
super().<strong>init</strong>()</p>
<pre><code>    """ Convolutional layer """
    self.b1 = batchnorm_relu(in_c)
    self.c1 = nn.Conv2d(in_c, out_c, kernel_size=3, padding=1, stride=stride)
    self.b2 = batchnorm_relu(out_c)
    self.c2 = nn.Conv2d(out_c, out_c, kernel_size=3, padding=1, stride=1)

    """ Shortcut Connection (Identity Mapping) """
    self.s = nn.Conv2d(in_c, out_c, kernel_size=1, padding=0, stride=stride)

def forward(self, inputs):
    x = self.b1(inputs)
    x = self.c1(x)
    x = self.b2(x)
    x = self.c2(x)
    s = self.s(inputs)

    skip = x + s
    return skip
</code></pre>
<p>class decoder_block(nn.Module):<br>
def <strong>init</strong>(self, in_c, out_c):<br>
super().<strong>init</strong>()</p>
<pre><code>    self.upsample = nn.Upsample(scale_factor=2, mode="bilinear", align_corners=True)
    self.r = residual_block(in_c+out_c, out_c)

def forward(self, inputs, skip):
    x = self.upsample(inputs)
    x = torch.cat([x, skip], axis=1)
    x = self.r(x)
    return x
</code></pre>
<p>class build_resunet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()</p>
<pre><code>    """ Encoder 1 """
    self.c11 = nn.Conv2d(3, 64, kernel_size=3, padding=1)
    self.br1 = batchnorm_relu(64)
    self.c12 = nn.Conv2d(64, 64, kernel_size=3, padding=1)
    self.c13 = nn.Conv2d(3, 64, kernel_size=1, padding=0)

    """ Encoder 2 and 3 """
    self.r2 = residual_block(64, 128, stride=2)
    self.r3 = residual_block(128, 256, stride=2)

    """ Bridge """
    self.r4 = residual_block(256, 512, stride=2)

    """ Decoder """
    self.d1 = decoder_block(512, 256)
    self.d2 = decoder_block(256, 128)
    self.d3 = decoder_block(128, 64)

    """ Output """
    self.output = nn.Conv2d(64, 1, kernel_size=1, padding=0)
    self.sigmoid = nn.Sigmoid()

def forward(self, inputs):
    """ Encoder 1 """
    x = self.c11(inputs)
    x = self.br1(x)
    x = self.c12(x)
    s = self.c13(inputs)
    skip1 = x + s

    """ Encoder 2 and 3 """
    skip2 = self.r2(skip1)
    skip3 = self.r3(skip2)

    """ Bridge """
    b = self.r4(skip3)

    """ Decoder """
    d1 = self.d1(b, skip3)
    d2 = self.d2(d1, skip2)
    d3 = self.d3(d2, skip1)

    """ output """
    output = self.output(d3)
    output = self.sigmoid(output)

    return output
</code></pre>

qq-me: <p>The model is okay. So something wrong with training code</p>
Accessing state_dicts in c++

SystemLord: <p>Hi,</p>
<p>In c++ how can I achieve the following (from the reinforcement learning tutorial):</p>
<p>`target_net_state_dict = [target_net.state_dict]<br>
policy_net_state_dict = [policy_net.state_dict]</p>
<p>for key in policy_net_state_dict:<br>
target_net_state_dict[key] = policy_net_state_dict[key]<em>TAU + target_net_state_dict[key]</em>(1-TAU)`</p>
<p>I’ve searched the forums but it almost seems like this isn’t supported?</p>
<p>Thanks</p>

SystemLord: <p>Hi guys,</p>
<p>Can anyone suggest a way to interpolate the weights of a network in c++ directly, such as this example in Python?</p>
<p>Thanks</p>

SystemLord: <p>Asking again - maybe <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

ptrblck: <p>I haven’t tried this code but something like this could work:</p>
<pre data-code-wrap="python"><code class="lang-python">const torch::OrderedDict&lt;std::string, at::Tensor&gt;&amp; model_params = model-&gt;named_parameters();
  std::vector&lt;std::string&gt; param_names;
  for (auto const&amp; w : model_params) {
    param_names.push_back(w.key());
  }

  torch::NoGradGuard no_grad;
  for (auto const&amp; w : weights) {
      std::string name = w.key().toStringRef();
      at::Tensor param = w.value().toTensor();

      if (std::find(param_names.begin(), param_names.end(), name) != param_names.end()){
        at::Tensor paramA = model_params.find(name);
        model_params.find(name)-&gt;copy_(paramA * TAU + param * (1 - TAU));
      } else {
        std::cout &lt;&lt; name &lt;&lt; " does not exist among model parameters." &lt;&lt; std::endl;
      };
</code></pre>

SystemLord: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Where is “weights” coming from here?</p>
Getting std::bad_alloc when loading the python tradined model in c++

Naseef: <p>Here is the python code :<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/c/4c64351ba5b8b679193c4d58afeb8bedb34260ab.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/4c64351ba5b8b679193c4d58afeb8bedb34260ab" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c64351ba5b8b679193c4d58afeb8bedb34260ab_2_690x243.jpeg" alt="image" data-base62-sha1="aTMYPR7JCOCiiWtSVUqtsEA6T59" width="690" height="243" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c64351ba5b8b679193c4d58afeb8bedb34260ab_2_690x243.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/c/4c64351ba5b8b679193c4d58afeb8bedb34260ab_2_1035x364.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/c/4c64351ba5b8b679193c4d58afeb8bedb34260ab.jpeg 2x" data-dominant-color="242525"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1081×382 66.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
cpp code :</p>
<pre><code class="lang-auto">#include &lt;iostream&gt;
#include &lt;memory&gt;
#include &lt;torch/script.h&gt;

int main()
{
	// Load the traced model
	torch::jit::script::Module module;

	// Load the saved model - update path to your .pt file
	module = torch::jit::load("net.pt");


	return 0;
}
</code></pre>
<p>cmakelists.txt :</p>
<pre><code class="lang-auto">cmake_minimum_required(VERSION 3.18)
project(torch_example)

include(FetchContent)

set(LIBTORCH_CPU_URL "https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-2.5.1%2Bcpu.zip")

FetchContent_Declare(
    Torch
    URL ${LIBTORCH_CPU_URL }
    DOWNLOAD_EXTRACT_TIMESTAMP TRUE
)
FetchContent_MakeAvailable(Torch)

add_executable(torch_example main.cpp)
target_link_libraries(torch_example PRIVATE "${TORCH_LIBRARIES}")
target_include_directories(torch_example PRIVATE "${torch_SOURCE_DIR}/include")
set_target_properties(torch_example PROPERTIES CXX_STANDARD 23)
</code></pre>
<p>error while running cpp :</p>
<p>terminate called after throwing an instance of ‘std::bad_alloc’<br>
what():  std::bad_alloc<br>
Aborted (core dumped)</p>
How I built my deep learning workstation

Roman_Glushko: <p>Recently I have managed to build my little deep learning workstation <img src="https://discuss.pytorch.org/images/emoji/apple/tada.png?v=9" title=":tada:" class="emoji" alt=":tada:"><br>
And I wanted to share my experience with you. So I wrote a blog post where tried to cover all aspects of building such a project including:</p>
<ul>
<li>reasons why I went for this</li>
<li>parts selection</li>
<li>hardware installation</li>
<li>software and CUDA installation</li>
<li>my current workflow</li>
</ul>
<p>I feel like this may be helpful for enthusiasts like myself who think about own GPU-accelerated machine:</p>
<p><a href="https://www.romaglushko.com/blog/how-i-built-my-ml-workstation/" rel="noopener nofollow ugc">https://www.romaglushko.com/blog/how-i-built-my-ml-workstation/</a> <img src="https://discuss.pytorch.org/images/emoji/apple/arrow_left.png?v=9" title=":arrow_left:" class="emoji" alt=":arrow_left:"></p>
<p>I hope this will be helpful. Let me know what do you think in the comments!</p>

Parveen_Kr90: <p>Hi Roman,</p>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/tada.png?v=12" title=":tada:" class="emoji" alt=":tada:" loading="lazy" width="20" height="20"> It’s great to see detailed insights on your build. If you ever need assistance with upgrades or optimizing your setup, here is my suggestion, you may check this website <a href="https://www.proxpc.com/" rel="noopener nofollow ugc">ProX PC</a>. They are providing customization options and custom build Pcs for <a href="https://www.proxpc.com/workstation/ai-deep-learning/ai-development" rel="noopener nofollow ugc">Deep Learning</a>.</p>
<p>Keep up the great work!</p>
Does loss.item() affect loss.backward()?

Michael_Zakhary: <p>Hello, just wanted to ask what is the correct order to call loss.item() and loss.backward(). I read a forum post saying that if you call item() after backward() execution is slower, however I also read that if you use .item() before backward() then the gradient does not flow. Can someone help me find a concrete answer for this?</p>

ptrblck: <p>Most likely the order won’t matter.<br>
Calling <code>item()</code> on a <code>CUDATensor</code> will synchronize your code and will thus potentially slow it down (e.g. if it wasn’t synchronized before anyway). I wouldn’t expect to see a large difference in speed if you synchronize the code before or after the <code>backward</code> call.</p>
<p>Calling <code>loss.item()</code> does not have any effect on the <code>loss.backward()</code> call unless you chain them <code>loss.item().backward()</code> which will raise an error.</p>

Michael_Zakhary: <p>So to make sure I understand correctly, there is no need to take care of their order, and if possible, removing .item() will speed up my training code? I don’t think I synchronize my code elsewhere, I have a very basic  training loop, although I am not completely aware of what synchronizing your code is exactly.</p>
Torchaudio.functional.speed very slow when called repeatedly using randomly generated factors

MikeK: <p>Hello, I’m trying to write a function that applies random augmentations to audio files, which has been converted to pytorch tensors in a prior operation. The problem is, the code I wrote runs really slow, I have located the culprit to be the “s” within x,_ = AF.speed(x, self.sr, s). If I change the “s” to a constant like 1.3 then the code runs swiftly with no problem. Has anyone encountered the same problem before? Does anyone know what the problem is? Or does anyone knows a better way to apply random audio augmentations? Any suggestions will help, thank you!</p>
<p>import torch as T<br>
import torchvision.transforms as VT<br>
import torchaudio.transforms as AT</p>
<p>class myAudioTransforms:<br>
def <strong>init</strong>(self, sample_rate:int, len:float, speed:tuple, noise:float):</p>
<pre><code>    self.sr = sample_rate
    self.speedgap = speed[1]-speed[0]
    self.speedmin = speed[0]
    self.spectrogram = AT.MelSpectrogram(sample_rate, 400, n_mels=96)
    self.resize = VT.Resize((96,96))

def get_params(self):
    s = random.random()*self.speedgap + self.speedmin
    s = round(s*10)/10
    return s

def __call__(self, x):
    x = x.to(T.float32).unsqueeze(0)
    s = self.get_params()
    x,_ = AF.speed(x, self.sr, s) # adjust speed
    spec = self.spectrogram(x)
    spec = self.resize(spec)
    return spec
</code></pre>
RuntimeError: derivative for aten::grid_sampler_3d_backward is not implemented (minimal reproducible example attached)

yunxiao: <p>Hello! So I’m trying to render a normal map from a density field. One of the ways to do this to grab the gradients of the density field w.r.t. to the point locations, and later the gradients participates in loss calculation. An example</p>
<pre><code class="lang-auto">import torch
import torch.nn.functional as F

feat = torch.randn(1, 1, 200, 200, 16) # features
inds = torch.rand(1, 1, 1, 420000, 3) # point locations

feat.requires_grad_(True)
inds.requires_grad_(True)

feat_ = F.grid_sample(feat, inds, 'bilinear', 'zeros', align_corners=True) # density field

gradients = torch.autograd.grad(feat_, inds, torch.ones_like(feat_), retain_graph=True, create_graph=True)[0] # (1, 1, 1, 420000, 3)

loss = gradients.mean() # using as an example

loss.backward()
</code></pre>
<p>which gives <code>RuntimeError: aten::grid_sampler_3d_backward is not implemented</code>. Essentially I need to grab such gradient, and to do that I’m backpropagating twice (I think). Wondering 1) if/when will this feature be supported? 2) there might be other ways to do what I’m trying to do that can work? Anyone has any ideas? Thanks</p>
<p><code>torch==2.1.2+cu121</code></p>

ehofgard: <p>Hi, I also need to backpropagate twice through grid_sampler_3d for a Hessian matrix calculation (it works in the 2D case but not in 3D). Did you find a fix for this? Thanks!</p>
AutoModelForCausalLM dataset process

hbao4148: <p>Essentially, I don’t understand how to train an autoregressive model, what the label should be, and what the loss function is. I found that the loss function hardly decreased during training, and I think it may be that the way I handle the label is wrong. Here is my code.</p>
<p>def preprocess_function(examples):<br>
model_inputs = tokenizer(<br>
examples[“text”],<br>
max_length=context_length,<br>
truncation=True,<br>
padding=True,<br>
return_tensors=‘pt’<br>
)<br>
labels = tokenizer(<br>
examples[“label”], max_length=context_length, truncation=True,padding=True,return_tensors=‘pt’<br>
)<br>
model_inputs[“labels”] = labels[“input_ids”]<br>
return model_inputs</p>
<p>tokenized_train = dataset_train.map(<br>
preprocess_function,<br>
batched=True,<br>
num_proc=4,<br>
remove_columns=dataset_train.column_names,<br>
)<br>
trainer = Trainer(<br>
model=model,<br>
args=training_args,<br>
train_dataset=tokenized_train,<br>
eval_dataset=tokenized_val,<br>
tokenizer=tokenizer,<br>
data_collator=data_collator,<br>
compute_metrics=compute_metrics,<br>
# preprocess_logits_for_metrics = preprocess_logits_for_metrics<br>
)<br>
I renamed the answer of the gsm8k dataset to label and the question to text</p>

Rajatavaa: <p>This is very unclear of what your hyper-parameters are</p>
How to properly unpickle a file?

Alibay_Alili: <p>I have done a small experiment:<br>
Using  <code>torch.save</code> and <code>torch.load</code> works as expected</p>
<pre><code class="lang-auto">l = [1, 2, 3]
torch.save(l, 'test.pkl')
l = torch.load('test.pkl') 
</code></pre>
<p>But when I try <code>torch.save</code> with <code>pickle.load</code> I got an error. Is it wrong approach?</p>
<pre><code class="lang-auto">l = [1, 2, 3]
torch.save(l, 'test.pkl')
with open('test.pkl', mode='rb') as f:
    l = pickle.load(f)
</code></pre>
<p>Traceback is:</p>
<pre><code class="lang-auto">UnpicklingError: A load persistent id instruction was encountered,
but no persistent_load function was specified.
</code></pre>
<p>Is it wrong to use <code>torch.save</code> with <code>pickle.load</code>?</p>
Teacher forcing ratio

MLangner: <p>Hello there,</p>
<p>In regard to training an RNN for NLP with LSTM cells, I wonder whether exposure bias is somehow changed when modifying the teacher forcing scheme.<br>
Do I choose per sequence (for the batch if you will) or per decoder iteration whether to use teacher forcing or not? the first would mean that some input sequences are fully teacher forced, while other sequences are processed with feeding in its own predictions only. In the latter version, the random variable for teacher forcing is inside the decoder iteration, which means that every sequence gets teacher-forced at some steps and not at other steps by proportion x.<br>
Any suggestions for the ratio? So far, I set it to 0.5, but before training at large, I wanted to ask whether this is adequate or whether I should choose a lower ratio.</p>
<p>Thanks in advance!</p>
Yolo Neural Layer

coded_bias_007: <p>Hey,<br>
I am trying to introduce a new layer to the vision yolov5 model by first defining the module class layer in the common.py file, calling it in the yolo.py and tf.py and yet I am getting the error below. Any leads on what I am missing would be greatly appreciated.</p>
<p>Thanks</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[384], line 7
      3 print('Printing the model architecture in the yaml configuration file.')
      5 cfg = 'models\sandbox_config.yaml'
----&gt; 7 model = Model(cfg, ch=3, nc = data_dict['nc'], anchors = hyp.get('anchors')).to(device)

File d:\new_pc_labwork\lab_work_projects\object_counting\training_yolov5\yolov5\models\yolo.py:238, in DetectionModel.__init__(self, cfg, ch, nc, anchors)
    236     LOGGER.info(f"Overriding model.yaml anchors with anchors={anchors}")
    237     self.yaml["anchors"] = round(anchors)  # override yaml value
--&gt; 238 self.model, self.save = parse_model(deepcopy(self.yaml), ch=[ch])  # model, savelist
    239 self.names = [str(i) for i in range(self.yaml["nc"])]  # default names
    240 self.inplace = self.yaml.get("inplace", True)

File d:\new_pc_labwork\lab_work_projects\object_counting\training_yolov5\yolov5\models\yolo.py:393, in parse_model(d, ch)
    390 no = na * (nc + 5)  # number of outputs = anchors * (classes + 5)
    392 layers, save, c2 = [], [], ch[-1]  # layers, savelist, ch out
--&gt; 393 for i, (f, n, m, args) in enumerate(d["backbone"] + d["head"]):  # from, number, module, args
    394     m = eval(m) if isinstance(m, str) else m  # eval strings
    395     for j, a in enumerate(args):

File &lt;string&gt;:1

NameError: name 'Involution' is not defined
</code></pre>
DataLoader weird IndexError

shibuina: <p>IndexError: Caught IndexError in DataLoader worker process 0.<br>
Original Traceback (most recent call last):<br>
File “/home/ducanh/anaconda3/envs/sigma_2/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py”, line 351, in _worker_loop<br>
data = fetcher.fetch(index)  # type: ignore[possibly-undefined]<br>
File “/home/ducanh/anaconda3/envs/sigma_2/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py”, line 55, in fetch<br>
return self.collate_fn(data)<br>
File “/tmp/ipykernel_8451/2647283629.py”, line 15, in <strong>call</strong><br>
img_ids = transposed_batch[2]<br>
IndexError: list index out of range</p>

ptrblck: <p>Are you using a custom <code>collate_fn</code> as <code>transposed_batch</code> is not used in the <a href="https://github.com/pytorch/pytorch/blob/09ba38c4b7860c181657db577a4ee9ea520f6285/torch/utils/data/_utils/collate.py">default implementation</a>?</p>

shibuina: <p>yes, I’m using collator = BatchCollator(32)</p>
Can I split my input in multiple embeddings? How would pytorch compute gradients?

Thomas_J: <p>Context: Learning pytorch, I’m trying to predict the next character given the past 1 character (Shakespeare input).  ; I’m aware that there are smarter ways to predict things but this is a good toy example to learn the ins and outs of pytorch.</p>
<p>At first, there is just 1 embedding layer (number of chars as input, number of chars as output) and a loss function. Training works: the loss function decreases with epochs.</p>
<pre><code class="lang-auto">def __init__(self):
  # vocab_size = len(set(chars in shakespear))) = 65
  self.embed = nn.Embedding(vocab_size, vocab_size)

def forward(self, x):
  return self.embedding(x)
</code></pre>
<p>Now I would like to predict the next character based on the past 2 chars.<br>
So my input is made of a tensor with 2 values. I aggrzgate the embeddings into a larger tensor and pass that into a prediction layer (linear).<br>
I do like so:</p>
<pre><code class="lang-auto">def __init__(self):
  # 10 is totally "random" 
  self.embed = nn.Embedding(vocab_size, 10)
  self.linear = nn.Linear(2 * 10, vocab_size)

def forward(self, x):
  e1 = self.embedding(x[0])
  e2 = self.embedding(x[1])
  e = torch.cat([e1, e2], dim=1)
  out = self.linear(e)
</code></pre>
<p>Now my question is:<br>
given that I’m not “chaining” layers properly (e.g with nn.Sequential), and that I’m “cutting” my input (<code>x[0], x[1]</code>) and reuniting it with <code>cat</code>, will gradients be computed correctly?<br>
I’ve read about Autograd but it’s unclear how it infers the graph and if “everything” is accepted.<br>
Thanks!</p>

ptrblck: <aside class="quote no-group" data-username="Thomas_J" data-post="1" data-topic="212676">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thomas_j/48/73841_2.png" class="avatar"> Thomas_J:</div>
<blockquote>
<p>given that I’m not “chaining” layers properly (e.g with nn.Sequential), and that I’m “cutting” my input (<code>x[0], x[1]</code>) and reuniting it with <code>cat</code>, will gradients be computed correctly?</p>
</blockquote>
</aside>
<p>Yes, since indexing and <code>torch.cat</code> are differentiable operations.</p>
Maybe there is a precision issue with torch.quantize_per_channel?

zfac: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/0/80f2c0d04c4f656428959b361ec665e34314eb4f.png" data-download-href="https://discuss.pytorch.org/uploads/default/80f2c0d04c4f656428959b361ec665e34314eb4f" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/0/80f2c0d04c4f656428959b361ec665e34314eb4f_2_690x358.png" alt="image" data-base62-sha1="ioJdaRx08NlsWwrKP2Dk7wWiBgb" width="690" height="358" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/0/80f2c0d04c4f656428959b361ec665e34314eb4f_2_690x358.png, https://discuss.pytorch.org/uploads/default/original/3X/8/0/80f2c0d04c4f656428959b361ec665e34314eb4f.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/0/80f2c0d04c4f656428959b361ec665e34314eb4f.png 2x" data-dominant-color="5E5E59"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">806×419 70.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>757.5/15 = 50.5 torch.round(50.5) = 50<br>
The result here should be 50, why is the actual output 51?</p>

yushu.gao: <p>The quantize_per_channel function uses std::lrint for rounding with the default round_mode, which rounds to the nearest integer. However, torch.round employs the half_to_even mode for rounding.</p>

zfac: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/1/711ffa5c7ab15c1409547f0b85c3d0e0ea1aa874.png" data-download-href="https://discuss.pytorch.org/uploads/default/711ffa5c7ab15c1409547f0b85c3d0e0ea1aa874" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/1/711ffa5c7ab15c1409547f0b85c3d0e0ea1aa874.png" alt="image" data-base62-sha1="g8KsVOxex4vLsiWv134RxobPYYA" width="690" height="456" data-dominant-color="F8F8F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1200×794 40.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The source code seems to use torch.round</p>

yushu.gao: <aside class="quote no-group" data-username="yushu.gao" data-post="2" data-topic="212389">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yushu.gao/48/50534_2.png" class="avatar"> yushu.gao:</div>
<blockquote>
<p>quantize_per_channel</p>
</blockquote>
</aside>
<p>Please refer to C++/cuda source code if you are in eager mode</p>
Confusion matrix problem

Manoj_Reddy: <p>i have a very confusion for my thesis code to prepare the confusion matrix i have done the feature extraction using pytorch code with transfer learning computer vision so my data is photographers identification using the convolutional neural networks using different resnets 18,34,50 i will upload the code with results here please someone help me to find out the confusion matrix or explain me how to do it on my own thanks in advance.</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/1BdRQGFeOEsfmh65L32hyh6Lb5QVtaX1u#scrollTo=r2g-A3UKvMWF">
  <header class="source">
      <img src="https://ssl.gstatic.com/colaboratory-static/common/8f9c73a58195bb6a1915ebabd5eedb19/img/favicon.ico" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1BdRQGFeOEsfmh65L32hyh6Lb5QVtaX1u#scrollTo=r2g-A3UKvMWF" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img width="260" height="260" src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar">

<h3><a href="https://colab.research.google.com/drive/1BdRQGFeOEsfmh65L32hyh6Lb5QVtaX1u#scrollTo=r2g-A3UKvMWF" target="_blank" rel="noopener nofollow ugc">Google Colab</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


ptrblck: <p>Assuming you are creating the model predictions and have the targets you could use <a href="https://scikit-learn.org/dev/modules/generated/sklearn.metrics.confusion_matrix.html"><code>sklearn.metrics.confusion_matrix</code></a> to create the confusion matrix.</p>

wilhelm: <p>Install <code>torchmetrics</code> and use the <a href="https://lightning.ai/docs/torchmetrics/stable/classification/confusion_matrix.html#id5" rel="noopener nofollow ugc">confusion_matrix function</a> available there.</p>

Manoj_Reddy: <p>I have understood the confusion matrix but i have 48 class names and the test data is with 800 images so for my transfer learning computer vision code i have given the num of images predicted as 6. iam confused now about how I can predict the true labels and predicted labels with it . my predicted images are 6 in these total 800 images how can i find out. if you can check the code and  any idea from your side would be helpful thanks</p>

ptrblck: <p>Usually you would not compute a confusion matrix for a single batch output, but would store the predictions of e.g. the validation set and compute the confusion matrix for the entire validation set.</p>
Search in documents

jsanchez: <p>Good day friends I have the case of 90000 documents in pdf format on which must be searched with keywords and as a result of the search referencing in which documents with their file name are the keywords.<br>
Do you suggest me to make a chat bot or what would be the solution being that every year the company is increasing more and more documents and if neural training is used then it should retrain the neurons to learn the new documents.<br>
Please support me</p>

Eduardo_Lawson: <p>I think a python script could handle it, because you are looking to do a search mission. You can build some loops to find this specific documents.</p>

rhady: <p>Have you considered “rga”? It is an extremely powerful tool deployed from the linux command line. It searches inside documents of many kinds including pdfs. It also does regex searches. You can specify which location(s) to search.<br>
You will need a little command line savvy to collect and to sort those files once identified.</p>
<p>rga: <a href="https://github.com/phiresky/ripgrep-all" class="inline-onebox" rel="noopener nofollow ugc">GitHub - phiresky/ripgrep-all: rga: ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc.</a></p>
Why tensor.to convert fp32 to fp8_e4m3=Nan if overflow

symbolics: <p>1、source code:<br>
bit_tensor = torch.tensor([0x7f7fffff, 0x7f800000, 0x7f800001], dtype = torch.uint32)<br>
src_tensor = bit_tensor.view(torch.float32).clone().to(device)<br>
dst_tensor = src_tensor.to(torch.float8_e4m3fn)</p>
<p>2、the result is:<br>
dst_tensor is tensor([[nan, nan, nan]], device=‘cuda:0’, dtype=torch.float8_e4m3fn)</p>
<p>3、cuda has satfinite to max_normal if overflow，behavior has different. any reason here?</p>

paulge: <p>When calling bit_tensor.view(torch.float32) you already have three numbers that cannot be represented in float32… The first one barely fits and the other two don’t fit anymore. Meaning you are moving this tensor <code>tensor([3.4028e+38, inf, nan])</code> to your cuda device. That’s the issue. If you call <code>tensor.view(torch.float32)</code> on your <code>uint32</code> tensor you are going to see this.</p>

symbolics: <p>sure, actually, I want to convert tensor([3.4028e+38, inf, nan], dtype = torch.float32) to tensor( dtype = torch.float8_e4m3fn),.<br>
but tensor.to() get the result tensor([[nan, nan, nan]], dtype=torch.float8_e4m3fn) in which nan=0x7F. this is diff from cuda-c with result: [ 3.4028e+38, inf] to max_normal=0x7E. any reason for this gap？</p>
Pytorch geometric temporal mamba installation

Charitini_S: <p>Hello,</p>
<p>I want to install Pytorch geometric temporal on an environment using the mamba commands. However I only found the pip version to install pytorch geometric temporal. Is there a conda/mamba possibilty?</p>

paulge: <p>Isn’t mamba a C++ version of conda? In conda once you’ve activated an environment you can also use pip install for packages unavailable with conda install. Maybe the same thing works in your mamba environment <a class="mention" href="/u/charitini_s">@Charitini_S</a> ?</p>

Charitini_S: <p>Okay, I am a bit worried about mixing mamba and pip commands. Here is how the installation would look:</p>
<ol>
<li>module load Mambaforge/23.3.1-1hpc1</li>
<li>mamba create -n GPT_env</li>
<li>mamba activate GPT_env</li>
<li>mamba install pytorch==2.2.2 torchvision=0.17.2 torchaudio==2.2.2 pytorch-cude=12.1 -c pytorch nvidia</li>
<li>mamba install pyg -c pyg</li>
<li><strong>pip install torch-geometric-temporal?</strong></li>
<li>mamba install spyder</li>
<li>…</li>
</ol>

paulge: <p>Yes, that’s how it works in conda. I would give it a shot and if not uninstall it again. I’d just watch out for torch-geometric-temporal overwriting your pytorch version, which I hope it doesn’t. Otherwise maybe reach out to the mamba devs on GitHub</p>
Self attention in pixelcnn

Vaibhav_Chahar: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2ce25b22d83276d78cd50dccb831a35641b58b8a.png" data-download-href="https://discuss.pytorch.org/uploads/default/2ce25b22d83276d78cd50dccb831a35641b58b8a" title="Screenshot from 2024-11-06 17-24-14"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2ce25b22d83276d78cd50dccb831a35641b58b8a_2_690x298.png" alt="Screenshot from 2024-11-06 17-24-14" data-base62-sha1="6p3YoNIFsTYZ1jNFX3VaXr21XBM" width="690" height="298" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/c/2ce25b22d83276d78cd50dccb831a35641b58b8a_2_690x298.png, https://discuss.pytorch.org/uploads/default/original/3X/2/c/2ce25b22d83276d78cd50dccb831a35641b58b8a.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/c/2ce25b22d83276d78cd50dccb831a35641b58b8a.png 2x" data-dominant-color="EEF3F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2024-11-06 17-24-14</span><span class="informations">722×312 20.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Hi all,<br>
I want to add the causal self attention block in my network (PixelCNN), but i can’t find any good material to start from. Recently I came across the picture shown, which have self attention mechanism. But I don’t know how to code it. If you help me, it is greatly appreciated. Please let me know where to start from and where to look like reference or some literature etc. Thanks in advance.</p>
<p>Cheers</p>
How to check gradients for ensemble-like architecture?

fabrizio_chavez: <p>Hello,</p>
<p>I am making an architecture with 2 Autoencoders. The input is passed to all auto encoders but only the autoencoder with the best (lowest) reconstruction is considered in the loss function. This should result in gradients update for that one autoencoder, no? How can I check the gradient flow in PyTorch for such a model? I have other things I want to check (e.g. are my gradients vanishing). Is PyTorch set up in a different way, such that I will need to use autograd to achieve this? ( I would rather not have to change anything with autograd)</p>
<p>I am new and have been looking around but have not found anything.<br>
Thank you!</p>

ruhaan10: <p>Hi, <a class="mention" href="/u/fabrizio_chavez">@fabrizio_chavez</a><br>
Can you provide a code, pseudocode, or something similar to clarify what you actually need?</p>

fabrizio_chavez: <p>Hi, thank you for getting back at me! I’ve actually figure this out in the time between the view and your response. I will close this thread.</p>

ruhaan10: <p>Ok, No Problem. It’s Nice to see that you figure out your self.</p>
Maintain output shape while using circular padding for CONVTRANSPOSE2D

lex2019: <p>Hello, since the circular padding is not support for <code>nn.CONVTRANSPOSE2D</code> so far, I use <code>F.pad</code> before the transpose convolutional layer to apply the padding operator. However, the output shape cannot be maintained as the input tensor, e.g., a tensor with shape <code>(16,16)</code> will be <code>(18,18)</code> after circular padding and cannot be shaped back to <code>(16,16)</code> using <code>nn.CONVTRANSPOSE2D</code>.</p>
<p>I searched some previous discussions and try to define output size by:</p>
<pre><code class="lang-auto">class MyConvTranspose2d(nn.Module):
    def __init__(self, conv, output_size):
        super(MyConvTranspose2d, self).__init__()
        self.output_size = output_size
        self.conv = conv
        self.p2d = (1, 1, 1, 1)
        
    def forward(self, x):
        x = F.pad(x, p2d, 'circular')
        x = self.conv(x, output_size=self.output_size)
        return x

conv = nn.ConvTranspose2d(1, 1, (3,3),stride=1)
self_layer = MyConvTranspose2d(conv, output_size=(16, 16))
tensor_16 = torch.randn(1, 1, 16, 16).to(device, dtype=torch.float)
self_layer(tensor_16)
</code></pre>
<p>but it pop put a error message: <code>ValueError: requested an output size of (16, 16), but valid sizes range from [20, 20] to [20, 20] (for an input of torch.Size([18, 18]))</code>. I wondering if there’s any suggestions on this problem. Thank you!</p>

yx_feng: <p><a href="https://github.com/pytorch/pytorch/issues/46027" class="inline-onebox" rel="noopener nofollow ugc">Padding mode for ConvTransposeNd · Issue #46027 · pytorch/pytorch · GitHub</a>.<br>
Not settled even in 2024. LOL.<br>
Could not believe they just support circular padding in conv2d, not in convtranspose2d.<br>
What amazing efficiency!</p>

ptrblck: <p>Contributions are more than welcome! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
Frontend C++ model loading error

complexhhs: <p>Hello, I’m testing my shared library(.so file) into other commercial program.</p>
<p>Problem is the commercial program shut down when it loads torch::jit type NN model.<br>
The program gives me unbelivable out of memory size error. For example, 14032033177600byte(14032GB).</p>
<p>Definitely, my shared library successfully loads NN model and runs the entire process when it is exported from a simple C++ executable file.</p>
<p>Surely, I inquired the problem to the commercial program’s engineer, and the answer is the commercial program is able to load pytorch model in the case of compilation using ABI shared libtorch version. Sadly, my server’s ABI version is too low to reproduce the model loading stage.</p>
<p>These are my server’s ABI information.</p>
<ul>
<li>GLIBCXX: 3.4 ~ 3.4.19</li>
<li>CXXABI: 1.3 ~ 1.3.7</li>
<li>GLIBC: 2.14, 2.2.5, 2.3, 2.3.2, 2.4</li>
</ul>
<p>To sum up, I compiled my shared library using torch-1.8.1 version and it runs well in the simple executable program, but it failed in the commercial program. I tried to compile my shared library using <a href="https://libtorch-cxx11-abi-shared-with-deps-1.8.1%2Bcpu.zip" rel="noopener nofollow ugc">libtorch_1.8.1_ABI</a>, but it shows library dependency not found error(GLIBC_2.18 not found).</p>
<p>Do anyone have idea what is the main reason of model running problem?</p>
Torch.nn.functional.embedding with output argument?

youkaichao1: <p>I’m looking into the function <a href="https://pytorch.org/docs/stable/generated/torch.nn.functional.embedding.html" class="inline-onebox" rel="noopener nofollow ugc">torch.nn.functional.embedding — PyTorch 2.5 documentation</a> , and want to have an <code>out</code> argument so that I can pass an existing buffer to it.</p>
<p>it looks strange why it does not have.</p>
<p>I can also use <code>torch.index_select(embedding.weight, dim=0, index=input, out=output_buffer)</code> since my index is only 1D. I’d like to know if pytorch has some existing solution though.</p>
Having problems Installing PyTorch

macro84: <p>I am tryinbg to install PyTorch on my new windows 11 machines with python 3.13 installed. I have an NVIDIA GPU version CUDA 2.15. I am unable to install.  The error I receive is for simple “pip install torch” as well as for<br>
“pip3 install torch torchvision torchaudio --index-url <a href="https://download.pytorch.org/whl/cu125" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu125</a>” is:</p>
<p>Defaulting to user installation because normal site-packages is not writeable<br>
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)<br>
ERROR: No matching distribution found for torch</p>

ptrblck: <p>PyTorch binaries for Windows do not support Python 3.13 yet as described in <a href="https://discuss.pytorch.org/t/unable-to-install-pytorch-on-python-3-13/212112/3">this post</a>.</p>
Error while while trying to train Tactron2 with a gpu

AndreeaV_13003: <p>Hi.I’ve been trying to train the Tacotron2 model for some time and an error keeps apearing.I don’t really know what to do .Please help<br>
This is the error that i am getting:<br>
RuntimeError: CUDA error: device-side assert triggered<br>
Compile with <code>TORCH_USE_CUDA_DSA</code> to enable device-side assertions.</p>
<p>When i use the debug it takes me to this line of code in torch modules :<br>
return self._apply(lambda t: t.type(dst_type))<br>
I am using gpu to train my model but i don’t know if it is a problem there</p>

ptrblck: <p>Rerun the code with blocking launches (via <code>CUDA_LAUNCH_BLOCKING=1</code>) to narrow down the failing line of code. Usually, an indexing operation is failing due to an input index being out of the valid range.</p>
TypeError: 'int' object is not callable for claculating training accuracy

amy2: <p>i am using a 3D cnn for feature extraction and svm as a classifier and I am trying to calculate the acuuracy of my training<br>
this is the logique for feature extraction and classification</p>
<pre><code class="lang-auto">def train_epoch(model, svm_classifier, train_loader, epoch,device):
    model.eval()  # Use ResNet in evaluation mode for feature extraction
    all_features = []
    all_labels = []

    for images, labels in train_loader:
        images = images.to(device)
        
        with torch.no_grad():  # Extract features without gradients
            features = model(images)
           
        all_features.append(features.cpu().numpy())
        all_labels.append(labels.cpu().numpy())

    # Convert features and labels to numpy arrays for SVM training
    all_features = np.concatenate(all_features, axis=0)
    all_labels = np.concatenate(all_labels, axis=0)

    # Shuffle and train SVM with extracted features
    all_features, all_labels = shuffle(all_features, all_labels)
    svm_classifier.fit(all_features, all_labels)

        # Predict on the training data
    train_predictions = svm_classifier.predict(all_features)
    
    # Calculate training accuracy
    train_accuracy = calculate_accuracy(all_labels, train_predictions)
    print(f"Epoch [{epoch}] - SVM training complete. Training accuracy: {train_accuracy * 100:.2f}%")
</code></pre>
<p>and for the accuracy calculation function i used this</p>
<pre><code class="lang-auto">`def calculate_accuracy(outputs, targets):
    with torch.no_grad():
        batch_size = targets.size(0)

        _, pred = outputs.topk(1, 1, largest=True, sorted=True)
        pred = pred.t()
        correct = pred.eq(targets.view(1, -1))
        n_correct_elems = correct.float().sum().item()

 return n_correct_elems / batch_size`
</code></pre>
<p>then this error occured</p>
<blockquote>
<p>batch_size = targets.size(0)<br>
^^^^^^^^^^^^^^^<br>
TypeError: ‘int’ object is not callable<br>
is it because I changed the tensors int np arrays ?</p>
</blockquote>

ptrblck: <aside class="quote no-group" data-username="amy2" data-post="1" data-topic="212609">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/amy2/48/62675_2.png" class="avatar"> amy2:</div>
<blockquote>
<p>is it because I changed the tensors int np arrays ?</p>
</blockquote>
</aside>
<p>Yes, as seen here:</p>
<pre data-code-wrap="python"><code class="lang-python">targets = torch.randn(10, 10)

# works
batch_size = targets.size(0)
print(batch_size)
# 10

# fails
targets = targets.numpy()
batch_size = targets.size(0)
# TypeError: 'int' object is not callable

# works but prints the number of elements
batch_size = targets.size
print(batch_size)
# 100

# works
batch_size = targets.shape[0]
print(batch_size)
# 10
</code></pre>
Torch compile with forward-mode automatic differentiation

SisyphusMountain: <p>Hello,<br>
I tried using the beta forward-mode automatic differentiation, but I ran into an issue when trying to compile my forward pass. I wonder if it was an error on my part or if it was an actual bug in the PyTorch code. Here is a minimal example that generates the error.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.autograd.forward_ad as fwAD

device = "cuda:0"

@torch.compile()
def step(model, x, device):
    x = x.to(device)
    tangent = torch.zeros_like(x, device=device)
    with fwAD.dual_level():
        dual_input = fwAD.make_dual(x, tangent)
        dual_output = model(dual_input)
    return None

lamb = 0.001  # Regularization parameter

x = torch.randn(2,3)
model = nn.Sequential(nn.Linear(3,3),
                      nn.BatchNorm1d(3),
                      nn.Linear(3,1)).to(device)
model.train()


step(model, x, device)
</code></pre>
<p>I get an error like such:</p>
<pre><code class="lang-auto">torch._dynamo.exc.TorchRuntimeError: Failed running call_function &lt;function batch_norm at 0x788edb8ac9a0&gt;(*(FakeTensor(..., device='cuda:0', size=(2, 3), grad_fn=&lt;AddmmBackward0&gt;,
           tangent=FakeTensor(..., device='cuda:0', size=(2, 3), grad_fn=&lt;AddBackward0&gt;)), FakeTensor(..., device='cuda:0', size=(3,)), FakeTensor(..., device='cuda:0', size=(3,)), FakeTensor(..., device='cuda:0', size=(3,), requires_grad=True), FakeTensor(..., device='cuda:0', size=(3,), requires_grad=True), True, 0.1, 1e-05), **{}):
InferenceMode::is_enabled() &amp;&amp; self.is_inference() INTERNAL ASSERT FAILED at "../aten/src/ATen/native/VariableMethodStubs.cpp":66, please report a bug to PyTorch. Expected this method to only be reached in inference mode and when all the inputs are inference tensors. You should NOT call this method directly as native::_fw_primal. Please use the dispatcher, i.e., at::_fw_primal. Please file an issue if you come across this error otherwise.

from user code:
   File "/home/enzo/Documents/git/LieEquiv/minimal_example.py", line 13, in step
    dual_output = model(dual_input)
  File "/home/enzo/mambaforge/envs/up_to_date/lib/python3.12/site-packages/torch/nn/modules/container.py", line 250, in forward
    input = module(input)
  File "/home/enzo/mambaforge/envs/up_to_date/lib/python3.12/site-packages/torch/nn/modules/batchnorm.py", line 193, in forward
    return F.batch_norm(
</code></pre>

ptrblck: <p>Would you mind creating an issue on <a href="https://github.com/pytorch/pytorch/issues">GitHub</a> if the code still fails in the latest nightly, please?</p>

SisyphusMountain: <p>Dear ptrblck,<br>
Thank you for your answer and your contributions to this community.<br>
The code does indeed fail with the latest nightly. I have created an issue on <a href="https://github.com/pytorch/pytorch/issues/139756" rel="noopener nofollow ugc">github</a></p>
RNN: Is it advisable initialize hidden state in training only once

Ras: <p>Is it better to initialize hidden state only once in a RNN training or initialize in every iteration of the training loop?<br>
For example, the example code here initialize hidden state in every iteration:<br>
<a href="https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial</a></p>
<pre><code class="lang-auto">def train(category_tensor, line_tensor):
    hidden = rnn.initHidden()

    rnn.zero_grad()

    for i in range(line_tensor.size()[0]):
        output, hidden = rnn(line_tensor[i], hidden)

    loss = criterion(output, category_tensor)
    loss.backward()

    # Add parameters' gradients to their values, multiplied by learning rate
    for p in rnn.parameters():
        p.data.add_(p.grad.data, alpha=-learning_rate)

    return output, loss.item()

## Training loop
for iter in range(1, n_iters + 1):
    category, line, category_tensor, line_tensor = randomTrainingExample()
    output, loss = train(category_tensor, line_tensor)
    current_loss += loss
</code></pre>
<p>For some other examples, the initialization happens outside the loop, and it keeps the previous value from the loop.</p>

Michael_Looney: <p>Think of the hidden state as the RNN’s “memory.”  It’s like a running record of what the network has seen so far in a sequence.  This memory is crucial because it allows the RNN to understand the context of the current input.<br>
Imagine reading a sentence: “The cat sat on the…”.  Your brain (like the hidden state) remembers “the cat sat on the” to predict the next word, maybe “mat” or “couch.”  If you forgot everything after each word, you’d have no idea what was going on!<br>
Initializing the hidden state at the start of each sequence is like starting a new sentence or paragraph. You need a fresh memory for each new piece of information.<br>
Now, that code example might be doing something different, like:</p>
<ul>
<li>Focusing on individual time steps: Perhaps it’s designed to analyze each input in isolation, without considering the surrounding context.</li>
<li>Illustrating a simplified scenario: It could be a basic example to demonstrate how RNNs work at a single time step level.<br>
But in most cases, you want to preserve the hidden state throughout a sequence to capture those all-important dependencies between time steps.  It’s like remembering the whole sentence to understand its meaning, not just individual words.</li>
</ul>