Sevelopment work in Docker?

username: Brent1
cooked: <p>Hello; a few getting started questions. Post is a bit rambling; so I numbered some questions; but any general advice on how to structure a development environment is most welcome.</p>
<p>In other projects; I do my development work in Docker. I find it nice to have the dockerfile to “document” the environment; and then <code>docker  run ... pytorch-dev /bin/bash</code> and from there tend to interactively work in the container. <strong>1.)</strong> Is that somewhat common for pytorch; or do most rely on the env management offered by Conda?</p>
<p>I started with the dockerfile given here - <a href="https://discuss.pytorch.org/t/building-from-source-in-docker-memory-use-requirments/133797/6" class="inline-onebox">Building from source in docker: Memory use requirments? - #6 by Padarn_Wilson</a> - which is pretty similar to the Dockerfile in the repo.</p>
<p>But running Coda in Docker seems to have some considerations</p>
<pre><code class="lang-auto">conda activate

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
```https://pythonspeed.com/articles/activate-conda-dockerfile/ has some pointers on using Conda; so perhaps either of the suggested options would be worth investigating.

</code></pre>
<pre><code class="lang-auto">printf '#include &lt;stdio.h&gt;\nint main() { printf("Hello World");}'|clang -x c -; ./a.out
bash: clang: command not found
bash: ./a.out: No such file or directory
</code></pre>
<p><strong>2.)</strong> Are these errors happening because the Conda environment isn’t active; so the required executables are not being found?</p>
<p>The below command did run through without any errors; so I do think the setup is quite close:</p>
<p><code>CMAKE_PREFIX_PATH="$(dirname $(which conda))/../" python setup.py develop</code></p>
<p>Trying to run python tests; I get the below</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "test/run_test.py", line 19, in &lt;module&gt;
    from torch.testing._internal.common_utils import (
  File "/home/brent/repos/pytorch/torch/testing/_internal/common_utils.py", line 46, in &lt;module&gt;
    import expecttest
ModuleNotFoundError: No module named 'expecttest'
</code></pre>
<p><strong>3.)</strong> If I try to stick with docker; I guess I could consider dropping using conda; since I could do multiple builds in different docker images.</p>
<p>Again; any pointers will be much appreciated!<br>
Thanks,Brent</p>

username: Padarn_Wilson
cooked: <p>Hey <a class="mention" href="/u/brent1">@Brent1</a>, I don’t have answers to all of your questions but I can give my perspective based on the workflow I’m using.</p>
<p>I didn’t intend to use conda (my thought was the same as your 3rd question), so there are some things missing that are in the original Dockerfile like</p>
<pre><code class="lang-auto">ENV PATH /opt/conda/bin:$PATH
</code></pre>
<p>I guess this is why you’re getting the conda setup error.</p>
<p>For the <code>expecttest</code> problem, this I’m not clear on at all, in my Dockerfile I have <code>pip install -r requirements.txt</code>, which should install this.</p>
<p>How are you triggering the tests? I just tried <a href="https://github.com/pytorch/pytorch/blob/master/test/test_torch.py" class="inline-onebox" rel="noopener nofollow ugc">pytorch/test_torch.py at master · pytorch/pytorch · GitHub</a> in my dev container and it was okay (but I haven’t rebuilt the container in a little while).</p>

username: ptrblck
cooked: <ol>
<li>
<p>You could install the conda binaries or pip wheels inside a docker container, so these approaches are not mutually exclusive.</p>
</li>
<li>
<p>However, it seems you cannot create a new conda env or activate it, so maybe you would need to run <code>conda init</code> or <code>conda init bash</code> first in your container?<br>
I guess <code>clang</code> is also not installed in the docker container, so you could add it to the <code>Dockerfile</code> and rebuild it.</p>
</li>
<li>
<p>Yes, I would prefer to use an “isolated” environment via a docker container (unless I’m debugging multiple binaries).</p>
</li>
</ol>

username: Brent1
cooked: <p>Thank you for the help; I’ve been able to get the tests to run. I could hopefully work out the answers to the below eventually on my own; I’ve still got a ways to go on my learning curve with the various components in use; so just posting here for completeness.</p>
<blockquote>
<p>Padarn_Wilson “I didn’t intend to use conda”</p>
</blockquote>
<p>Are the <code>conda</code> commands in the dockerfile in <a>building-from-source-in-docker</a> “just” to be able to use conda packages - such as <code>conda-build; pyyaml; numpy; ipython</code> - but not to use an actual conda environment?</p>
<blockquote>
<p>pip install</p>
</blockquote>
<p>After running</p>
<pre><code class="lang-auto">pip install -r requirements.txt
pip install pytest
</code></pre>
<p><code>python test/run_test.py</code> starts running without errors. YAY. I’m running it from a shell which is connected to the running container.</p>

username: Padarn_Wilson
cooked: <p>Sorry I wasn’t notified of your reply!</p>
<blockquote>
<p>Are the <code>conda</code> commands in the dockerfile in building-from-source-in-docker “just” to be able to use conda packages - such as <code>conda-build; pyyaml; numpy; ipython</code> - but not to use an actual conda environment?</p>
</blockquote>
<p>Right, I don’t have a conda environment separate from the global (or user) python stup.</p>