Can someone provide the steps to upload my pytorch project on Google Colab?

krishna511: <p>Hi all,<br>
I want to run my project on google colab, as dnt have GPU facility. Uptill now I have uploaded the complete project file in my drive. What should be next, Plz give me a step wise process . My project has several parts (.py files) and dataset too.</p>
<p>Regards</p>

Henry_Chibueze: <p>Compress the project folder to a zip or RAR file</p>
<p>Start up google colab and select the add file icon by the left plane (the plane may be collapsed)</p>
<p>Upload the compressed file to the colab</p>
<p>Then install unrar package (Linux version) in colab using the terminal (always remember to add ‘!’ before a code to tell the colab terminal that it’s a terminal command)eg: ‘!apt-get install unrar’</p>
<p>Then after installing unpack the compressed file by the command ‘!unrar X rarfile’</p>
<p>And u r done <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

krishna511: <p>Well Thank for reply , here I am not able to find add file icon, This is the left plane. When I open colab there is a pop up window, having options like github&gt;&gt;&gt;upload etc.</p>
<p>And actually I have done uploading in drive already. (Not rar but whole folder)<br>
Can I upload this into colab now?<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/c/0c56adccb4070a2145b77da38254caae5f58bcfb.png" alt="colab" data-base62-sha1="1L9raZgnR2h7q3BM90FKZ8vjQDx" width="411" height="385"></p>

Henry_Chibueze: <p>Click on the folder icon (the last icon by the left side)</p>

Henry_Chibueze: <p>If ur folder is already in ur drive then u can do as this link says:<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://www.marktechpost.com/wp-content/uploads/2017/06/Untitled-design-3-300x300.png" class="site-icon" width="300" height="300">
      <a href="https://www.marktechpost.com/2019/06/07/how-to-connect-google-colab-with-google-drive/" target="_blank" rel="noopener nofollow ugc" title="10:19PM - 07 June 2019">MarkTechPost – 7 Jun 19</a>
  </header>
  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/460;"><img src="https://i1.wp.com/www.marktechpost.com/wp-content/uploads/2019/06/developer-3461405_1920.png?fit=1920%2C1280&amp;ssl=1" class="thumbnail" width="690" height="460"></div>

<h3><a href="https://www.marktechpost.com/2019/06/07/how-to-connect-google-colab-with-google-drive/" target="_blank" rel="noopener nofollow ugc">How to Connect Google Colab with Google Drive | MarkTechPost</a></h3>

<p>How to Connect Google Colab with Google Drive. In this tutorial, you’ll learn how to connect your Google Colab with Google Drive to build some Deep Learning</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


krishna511: <p><a class="mention" href="/u/henry_chibueze">@Henry_Chibueze</a><br>
Thanks again, Just done<br>
and now in that extream left panel i  can find all those python files and relevant dataset. after mounting the drive. Now I am confused do when I double click there, it is open in right panel. Now my question is do I need to copy this to jupyter cells here or is there any other way?</p>
<p>Regards</p>

Henry_Chibueze: <p>U mean copying the python code to jupyter cell?<br>
If that’s what u mean then no you don’t need to copy the python code to the cell</p>
<p>If u want to run code, what u need to do is to run on the terminal with python.<br>
If u are familiar with Linux terminal or cmd, all u need to to is to navigate to the dir where the code is like this <code>!cd dir</code> then u run the command <code>!python code.py</code>.</p>
<p>Don’t forget to add the ‘!’ b4 a terminal command else the command will attempt to run as a python code which will throw an error.</p>
<p>If u want to edit your code, then all u need to do is click on the python file on the left panel and a text editing plane will open on the right where u can edit the code.</p>
<p>Hope this helps you <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

krishna511: <p>I am sorry I may sound like a fool here, but I wanted to know how to run this project on co lab, as the issue on my local machine is GPU acceleration problem.<br>
Or are you saying after this uploading on colab the compilation done through command line?<br>
I am still confused<br>
Sorry</p>

Henry_Chibueze: <p>Sorry for the late reply <img src="https://discuss.pytorch.org/images/emoji/apple/pray.png?v=9" title=":pray:" class="emoji" alt=":pray:"></p>
<p>I don’t know about you but when I program, I like to run my code on the terminal of the operating system I’m using windows or Linux and not on the IDE (this is just my preference)</p>
<p>So it’s kinda similar to colab.<br>
Colab uses Linux as it’s operating system and the a python version is installed in it by default with some Machine learning modules and others.</p>
<p>Normally in any operating system u are using, if u want to run a code from the terminal all u simply need to do is open the terminal and type the command <code>cd 'code dir'</code> to navigate to the code directory and then <code>python code.py</code> to run it. This is no different with colab.<br>
Once u upload ur code on colab whether from Google drive or from local directory, it is stored in the current working directory EG;<br>
Assuming u uploaded a folder named ‘project’ and this folder has ur code named ‘project_class.py’, all u need to do is to type <code>!cd project</code> to navigate into ‘project’ folder and then u type <code>!python project_class.py</code> to run the code on colab.<br>
If u want to edit the code, simple go to the left panel and click on particular file u wish to edit and a text editor will appear on the right side.</p>
<p>Hope this helps</p>

Henry_Chibueze: <p>So to answer ur questions, yes the code execution is done via the command terminal and u don’t need to do anything special or navigate to some unknown place to open a terminal.<br>
All u simplly need to do is to type the command on a cell like how u’ll type a python code in a cell but when u r typing a terminal command u need to add ‘!’ before the command. This will make the terminal differentiate terminal command from python code.</p>
<p>U grab?<img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

krishna511: <p>I really find it very straight forward and clear so accepted it as answer for other readers. But still for me its not working.<br>
I am doing exactly same  but it says NO such file or  directory  as you can see its there in drive.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/9/1911141539073e17fec0d65af7070ba84109d9d5.png" data-download-href="https://discuss.pytorch.org/uploads/default/1911141539073e17fec0d65af7070ba84109d9d5" title="colab"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/1911141539073e17fec0d65af7070ba84109d9d5_2_690x336.png" alt="colab" data-base62-sha1="3zKwGr9JZCUIpWyTxFNBYxXzEpL" width="690" height="336" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/1911141539073e17fec0d65af7070ba84109d9d5_2_690x336.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/9/1911141539073e17fec0d65af7070ba84109d9d5_2_1035x504.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/9/1911141539073e17fec0d65af7070ba84109d9d5.png 2x" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">colab</span><span class="informations">1127×550 35.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Thank you for helping</p>
<p>Regards</p>

Henry_Chibueze: <p>Type <code>!dir</code> and run the command<br>
If u see all the files and directories of ‘CNN_as_MATLAB’ folder listed as the list of directories on the terminal, then that directory is ur working directory.<br>
So so if u are already in that working directory, u don’t need to run the <code>!cd.....</code> command anymore. All u need to do is to just run the python code:</p>
<pre><code class="lang-auto">!python EmoDB_1.py
</code></pre>

krishna511: <p>Cant thank you enough,<br>
Just one more question how to set device here in colab</p>
<p>model=model.to(device)<br>
?</p>

Henry_Chibueze: <p>Yes<br>
The same way u always do.<br>
Tho it’s preferable that u use the cpu environment on colab rather than the gpu environment except u are using colab pro (which is a paid version)<br>
The gpu environment in colab can be really unstable sometimes, but u can always try it out.</p>

krishna511: <p>Actually my question is what should I take as device</p>
<p>device = ‘torch.cuda.device’<br>
model = model.to(device)</p>
<p>??/<br>
its not working</p>

Henry_Chibueze: <p>To switch processor environments all u need to do is go to the top bar and select ‘runtime’, then select ‘change runtime type’ and u’ll see the one u are currently using (‘none’) and u can switch by selecting GPU or TPU on the drop down.<br>
Remember that when u change runtime u need to upload your project folder again coz it’s like switching to a different computer</p>
<p>If u want to use the TPU tho u have to install ‘torch.xlsa’ package. U can just search for ‘how to use run pytorch on TPU’.</p>
<p>The code for specifying running device should be still be kept<br>
<img src="https://discuss.pytorch.org/images/emoji/apple/upside_down_face.png?v=9" title=":upside_down_face:" class="emoji only-emoji" alt=":upside_down_face:"></p>

krishna511: <p>I am trying to install torch_xla as</p>
<p>!pip install torch_xla</p>
<p>But the error is</p>
<p>ERROR: Could not find a version that satisfies the requirement torch_xla (from versions: none)<br>
ERROR: No matching distribution found for torch_xla<br>
<img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji only-emoji" alt=":frowning:"></p>

Henry_Chibueze: <p>LOL <img src="https://discuss.pytorch.org/images/emoji/apple/rofl.png?v=9" title=":rofl:" class="emoji" alt=":rofl:"> <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=9" title=":joy:" class="emoji" alt=":joy:"> it’s really funny coz I experienced ur problems too and in the exact order as urs .<br>
Don’t mind me🙂</p>
<p>Anyways just look at this site for details on that:<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20201104-085601-RC00_340647971" class="site-icon" width="128" height="128">
      <a href="https://colab.research.google.com/github/pytorch/xla/blob/master/contrib/colab/getting-started.ipynb" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="256" height="256">

<h3><a href="https://colab.research.google.com/github/pytorch/xla/blob/master/contrib/colab/getting-started.ipynb" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


Mughees: <p>Hi,<br>
Amazing discussion going on.</p>
<p>Just to mention that its advisable to use <code>%run script.py</code> on colab instaed of  <code>!python script.py</code>. see <a href="https://github.com/googlecolab/colabtools/issues/1067" rel="noopener nofollow ugc">this</a>.</p>
<p>Best,<br>
Mughees</p>

Henry_Chibueze: <p>Ok thanks alot🙃<br>
We learn everyday don’t we?</p>