3D object reconstruction Dataset , model problem

username: Ashish_Gupta1
cooked: <p>Hello Everone. I am working on 3D image reconstruction using a single image. Now<br>
I have trained my model but it’s not working. Due to the lack of guidance in 3D, I can’t figure out the error. Please help me if there is some error in my approach to training.</p>
<p>First, let me explain what I am following.</p>
<p>1.I have taken synthetic dataset(I am uploading the mini dataset for 1 type of object per class).<br>
Dataset-<br>
<a href="https://drive.google.com/open?id=1AMjKQUujUlNSRRqnoKsJSteNed9mTEiI" class="onebox" target="_blank" rel="noopener nofollow ugc">https://drive.google.com/open?id=1AMjKQUujUlNSRRqnoKsJSteNed9mTEiI</a></p>
<p>2.I have taken 4 classes(objects) with 100 different types of objects per class.</p>
<ol start="2">
<li>Each object is having 24 images where all these 24 images have one binvox file.</li>
</ol>
<p>3.so I have associate each image with its 3D model separately.</p>
<p>For eg -&gt;we have one chair class.Lets take a type of chair which have 24 images and one 3D model.</p>
<p>So I associate each of the image with 3D model.</p>
<p>4.Now I put them into traning with a batch size of 24 without shuffling.</p>
<p>5.Learning rate,loss is according to your paper.</p>
<p>6.after training to 50 epochs my average loss reduces from 11.92 to 4.76.</p>
<p>As I check after training, my model is outputting the same output for all the classes.</p>
<p>7.i have also used batch normalisation with Xavier initialisation.</p>
<p>8.after training to test I have taken argmax(returns indices of max value) along the depth or channel dimensions(dim=1).</p>
<p>My input shape-[400,24,128,128,3] to the model.</p>
<p>(Here 400 means 4 classes *100 different objects per class)</p>
<p>Output by model something like-[24,2,32,32,32]</p>
<p>After argmax -[24,32,32,32]</p>
<p>This is all I have done.</p>
<p>Please rectify me if I have done something wrong in preparation of dataset(or I have to extend my dataset by getting more images or augmentation). Or there is a different way to train these types of models. Or at argmax.</p>
<p>Here is the given model and training code–</p>
<pre><code class="lang-auto">n_epochs = 10
#model=model.cuda()
model.train()
train_loss = 0.0
for epoch in range(1, n_epochs+1):
    print("number of epoch" ,epoch)
    print("ALL ABOUT LOSS(training)--------",(train_loss/(500)),"----------/n")
    train_loss = 0.0 

    model.train()
    for i in range(len(arr)):
                  
                  data=arr[i].cuda()
                  tar=arr_3d[i].cuda()
                  tar=tar.long()
                  optimizer.zero_grad()
                   
                  output=model(data)
                  
                  loss = criterion(output, tar)
                  loss.backward()
                    # perform a single optimization step (parameter update)
                  optimizer.step()
                    # update training loss
                  train_loss += loss.item()*data.size(0)
                  
                  if(i%100==0 and i!=0):
                    scheduler.step()
                    print(scheduler.get_lr())


    
</code></pre>
<p>Model –<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/7/7ea81ea7ffc36816dc2b26574fa6bd08cec9e98d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/7ea81ea7ffc36816dc2b26574fa6bd08cec9e98d" title="Screenshot_20190312-115931-1"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/7/7ea81ea7ffc36816dc2b26574fa6bd08cec9e98d_2_200x300.jpeg" alt="Screenshot_20190312-115931-1" data-base62-sha1="i4sm5a6co6A23XexdWG9hdzVM5f" width="200" height="300" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/7/7ea81ea7ffc36816dc2b26574fa6bd08cec9e98d_2_200x300.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/7/7ea81ea7ffc36816dc2b26574fa6bd08cec9e98d_2_300x450.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/7/7ea81ea7ffc36816dc2b26574fa6bd08cec9e98d_2_400x600.jpeg 2x" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_20190312-115931-1</span><span class="informations">1082×1640 111 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/a/a3304077125652d773e1dfb5ab7b7ea200b8c5e0.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/a3304077125652d773e1dfb5ab7b7ea200b8c5e0" title="Screenshot_20190313-002300-2"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/a/a3304077125652d773e1dfb5ab7b7ea200b8c5e0_2_690x228.jpeg" alt="Screenshot_20190313-002300-2" data-base62-sha1="nhDcOE8M71akpwj5Up2M2mP3pja" width="690" height="228" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/a/a3304077125652d773e1dfb5ab7b7ea200b8c5e0_2_690x228.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/a/a3304077125652d773e1dfb5ab7b7ea200b8c5e0_2_1035x342.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/a/a3304077125652d773e1dfb5ab7b7ea200b8c5e0.jpeg 2x" data-dominant-color="EAEAE9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_20190313-002300-2</span><span class="informations">1080×358 63.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Ashish_Gupta1
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Please help if possible<br>
I have also redesigned the dataset by taking 4 classes with 25 different objects each (shape Net dataset). And use augmentation by different transformations to get around 4 different images for each image per object per class.</p>

username: ptrblck
cooked: <p>The code looks generally alright.<br>
Which criterion are you using and what is the last non-linearity in your model?</p>
<p>PS: Tagging certain people might discourage others to answer in your thread and as you can see I’m quite clueless here. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: Ashish_Gupta1
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> thanks for your reply.<br>
I am using Cross Entropy Loss as criterion. Last non Linearity in my model is <code>Softmax</code> across <code>dimension=1</code> ie. across the channels[else i can say across the depth].<br>
In my case my model outputs<code>[batch_size,2,32,32,32]</code>. So i took softmax across <code>dim=1</code>.</p>
<p>PS: Your answers always helps.And this gives me motivation to carry further my work.Thanks to you.<img src="https://discuss.pytorch.org/images/emoji/apple/innocent.png?v=9" title=":innocent:" class="emoji" alt=":innocent:"></p>

username: ptrblck
cooked: <p><code>nn.CrossEntropyLoss</code> expects raw logits, so remove the softmax at the output of your model and try to train it again.</p>
<p>That’s good to hear and you are most welcome! <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: Ashish_Gupta1
cooked: <p>Yeh I will try it. Thanks</p>

username: Muhammad_Wajahat
cooked: <p>Hey <a class="mention" href="/u/ashish_gupta1">@Ashish_Gupta1</a> I am working on the same problem if you don’t mind  can you share the code you used to load and render 3d models in python</p>

username: Ashish_Gupta1
cooked: <p><a class="mention" href="/u/muhammad_wajahat">@Muhammad_Wajahat</a> For my problem I have used pascal voc and shapeNet. The shapeNet dataset 3D models comes in .binvox format and the link for thr binvox lib is… <a href="https://github.com/dimatura/binvox-rw-py" rel="nofollow noopener">https://github.com/dimatura/binvox-rw-py</a></p>
<p>Let’s connect via linkdin-- <a href="https://www.linkedin.com/in/ashish-gupta-36934874" rel="nofollow noopener">https://www.linkedin.com/in/ashish-gupta-36934874</a></p>

username: Muhammad_Wajahat
cooked: <p><a class="mention" href="/u/ashish_gupta1">@Ashish_Gupta1</a> sure lets connect via linkedin: <a href="https://www.linkedin.com/in/muhammad-wajahat-614249118/" rel="nofollow noopener">https://www.linkedin.com/in/muhammad-wajahat-614249118/</a><br>
and I am also using shapeNet but I am using obj files because I want to use point cloud for my neural network and meshes to render images of shapeNet.<br>
Also I want to know how you rendered the images because I am using blender api to render images but it is messing up the textures of my renders</p>