akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
AttributeError: 'RNN' object has no attribute 'n_layers'

username: av9555
cooked: <p>Greetings PyTorch Community,</p>
<p>I am trying to implement an RNN model using GRU cell. Look for some guidance on why the init_hidden() function returning the error.</p>
<pre><code class="lang-auto">class RNN(nn.Module):
    def __init__(
        self, 
        batch_size,
        e_dim = 200, 
        n_hidden = 6, 
        n_class = 6,
        n_layers = 1
    ):
        super(RNN, self).__init__()
        self.n_layers = n_layers
        self.e_dim = e_dim
        self.n_hidden = n_hidden
        self.batch_size = batch_size
        self.n_class = n_class
        
        self.gru = nn.GRU(
            self.e_dim, 
            self.n_hidden,
            num_layers = n_layers,
            batch_first = True
        )
        self.fc = nn.Linear(self.n_hidden, self.n_class)

    def init_hidden(self):
        return torch.randn(self.n_layers, self.batch_size, self.hidden_size)

    def forward(self, inputs):
        # Avoid breaking if the last batch has a different size
        batch_size = inputs.size(0)
        if batch_size != self.batch_size:
            self.batch_size = batch_size
            
        output, hidden = self.gru(inputs, self.init_hidden())
        output = self.fc(output[:, :, -1]).squeeze()
        return output
</code></pre>

username: ptrblck
cooked: <p>I don’t get your mentioned error message, but instead <code>self.hidden_size</code> is undefined.<br>
I assume you meant to use <code>self.n_hidden</code> instead.</p>
I suggest to do more courses with Coursera rather than Udacity for the situation that we do not have enough tutorial on Pytorch

username: JonathanSum
cooked: <p>I don’t think Udacity and Fast.ai are going to work because of the problems I have mentioned in below:</p>
<ol>
<li>
<p>For the Udacity, it only covers some very basic ideas or not even basic. For example, the word embedding only teaches the index to the word. If the student wants to learn more, he or she need to pay 600 dollars or 400 dollars per month.</p>
</li>
<li>
<p>For the fast.ai, it pretty much teaches how to use their library instead of how to use PyTorch.</p>
</li>
<li>
<p>For the database on privacy on Udcaity with FB, that one is not even about deep learning.</p>
</li>
</ol>
<p>On the Coursera site, I have taken a lot of courses from them in Tensorflow on NPL(not just index to word), CNN, DNN, Time series prediction, Javascript in Tensorflow, and Tensorflow Quantization. I have not paid a single dime on them because I applied Finical Aid, and I have built a lot of projects and earned many certificates from them. One of the good advances of coursera is they also have many courses about the theory of deep learning by Andrew Ng. If they have a good Pytorch class with FaceBook, I am sure many of people or students can have advantages from that.</p>
Torchvision Models

username: Epoching
cooked: <p>I see lots of models for <a href="https://pytorch.org/docs/stable/torchvision/models.html#classification" rel="nofollow noopener">Classification</a>, but not too many for <a href="https://pytorch.org/docs/stable/torchvision/models.html#semantic-segmentation" rel="nofollow noopener">Semantic Segmentation</a>.</p>
<p>Just curious if anyone has an idea for why things like <a href="https://arxiv.org/abs/1505.04597" rel="nofollow noopener">U-Nets</a> aren’t included in the torchvision model zoo. I have always considered U-Nets to be a classic, like the “ResNets of semantic segmentation”. Is a DeepLab3 or FCN just ultimately better, or perhaps torchvision is still being updated? Maybe U-Nets aren’t typically used for transfer learning/feature extractors (compared to ResNets, or even the R-CNN family of object detection/instance segmentation/keypoint estimation models)?</p>
Combining image and scalar value as inputs

username: talkingtoaj
cooked: <p>I have been training a resnet34 Convolutional network using fastai which classifies images of rooms of a house as to ‘kitchen’,‘bathroom’,‘bedroom’ etc. I am trying to predict the real estate selling price of this property from the photos. I have a seperate NN which does reasonably well using other advertisement data, but I want to see how the photos can add to this.</p>
<p>Hence, I can provide the initial price estimate as a scalar value in parallel to the photo of the room, using my previously trained Convolution NN as a starting point (transfer learning)</p>
<p>I’m new to Pytorch and trying to get a better grasp at it as I go deeper than what fastai provides. How would I define my nn.Module class to receive an image and scalar value as the inputs, then forward the scalar value past the Convolutional resnet34 to be combined with it’s top layer for a final 1 or 2 layer before attempting an improved final price prediction scalar output?</p>

username: aauker
cooked: <p>Great question- covariates indeed can be useful, but also tricky to implement correctly.  I assume you have a nn.Module class that includes resnet34 as one of its members with a few linear layers. Something like:</p>
<pre><code class="lang-auto">self.resnet = resnet34()
self.regression = nn.Linear(1000,1)
</code></pre>
<p>to predict (I’m guessing) price (output of self.regression).</p>
<p>It’s actually quite easy in principle to do what you want, but the implementation might take some optimization.  Generally, you need to pass both the input image (as a 4D tensor) and price (as a 2D tensor) to the forward method of your module class.  Resnet outputs 1000 linear dimensions or features, so after this you can concat the price as a feature to this output.</p>
<p>Though there are some considerations - for example, it probably will be best to reduce the dimensionality a bit before adding the price into the final layers.  Additionally, you need to think about normalization of the price feature, is it from [-1,1], [0,1]?  Or perhaps a bit more complicated, perhaps as a on-hot encoded price of 0 or 1, or a couple bits… it depends on your use-case.  Though if you have a NN output, you can try just passing that straight through and see what happens first.</p>
<p>Here’s a general scheme –</p>
<pre><code class="lang-auto"># Your resnet module
self.resnet = resnet34()

# Reduce dimension from resnet output 
self.reduce = nn.Linear(1000,100)

# Some final regression layer
self.regression = nn.Linear(101,1)

def forward(input_image, price):
    # Output from resnet has shape (N, 1000)
    out = self.resnet(input_image)

    # Reduce dimension, now has shape (N, 100)
    out = F.relu(self.reduce(out))
    
    # Concat price along feature dimension, shape (N, 100+1)
    out = torch.cat([out, price], dim=1)
    
    # Some final regression 
    out = self.regression()
    return out
</code></pre>