Thoughts on Visual Development

chenglu: <p>I think machine learning applications, especially deep learning, can be abstracted as data processing. If that’s true, can we do some visualization for the data processing pipeline, like a graphic workflow configuration tool to build the training and validation process? I’ve seen that Microsoft already has it in their <a href="https://docs.microsoft.com/en-us/azure/machine-learning/service/ui-concept-visual-interface" rel="noopener nofollow ugc">Azure</a> . And the <a href="https://developer.nvidia.com/digits" rel="noopener nofollow ugc">NVIDIA DIGITS</a> seems also want’s to do this.</p>
<p>Recently I’ve been working on this for a time but I’ve met a lot of troubles. I want to keep the flexibility as well as the simplicity, but it seems like very hard to keep both of them. And I start to doubt is this the correct way, or should we just write plain code?</p>
<p>I want to know what the community sees about this, so if you have any ideas, positive or negative, please let me know : )</p>
<p>Here are some screenshots about these kind of products:</p>
<p>Microsoft Azure:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/c/c6b4332325a77c000deb32532159f40dbed86995.png" data-download-href="https://discuss.pytorch.org/uploads/default/c6b4332325a77c000deb32532159f40dbed86995" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/c/c6b4332325a77c000deb32532159f40dbed86995_2_438x500.png" alt="image" data-base62-sha1="slOC9nB0p5p29E4jJfL4ZIE0FI9" width="438" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/c/c6b4332325a77c000deb32532159f40dbed86995_2_438x500.png, https://discuss.pytorch.org/uploads/default/optimized/2X/c/c6b4332325a77c000deb32532159f40dbed86995_2_657x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/c/c6b4332325a77c000deb32532159f40dbed86995_2_876x1000.png 2x" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">954×1088 94.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>NVIDIA DIGITS<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/5297ed596679f65b62e17045fe4212251733366c.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/5297ed596679f65b62e17045fe4212251733366c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/5297ed596679f65b62e17045fe4212251733366c_2_690x491.jpeg" alt="image" data-base62-sha1="bMEEBLaOSOFstyVOFmicQt5Z3HK" width="690" height="491" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/5297ed596679f65b62e17045fe4212251733366c_2_690x491.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/5/5297ed596679f65b62e17045fe4212251733366c_2_1035x736.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/5/5297ed596679f65b62e17045fe4212251733366c_2_1380x982.jpeg 2x" data-dominant-color="E3E3E3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1732×1234 397 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Both of them are hard to use, I’d rather write plain code : (</p>

Tony-Y: <p>r/MachineLearning is the best place for your discussion:</p>
<p><a href="https://www.reddit.com/r/MachineLearning/" class="onebox" target="_blank" rel="nofollow noopener">https://www.reddit.com/r/MachineLearning/</a></p>

jinfagang: <p>Can agree with you anymore.</p>
<p>those graphics tools only help for newbies but for masters it’s not needed at all.</p>

chenglu: <p>Have already posted there <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

chenglu: <p>I think you’re right for the 2 productions(Azure and NVIDIA DIGITS), I have tried them and I’m confused about WHO will use these tools:</p>
<ul>
<li>For developers? Not flexible enough to build complicated applications.</li>
<li>For newbies? Too much to learn and also, not flexible enough.</li>
</ul>
<p>So the baseline is, the machine learning applications require coding abilities. But is it possible to visualize the process, or even build the process by semi-graph-semi-code way?</p>

Tony-Y: <p>Cloud you give me the link to your post?</p>

chenglu: <p>Story is, last year I have developed a <a href="https://github.com/minetorch/minetorch" rel="nofollow noopener">tool</a> to help people use PyTorch more easily. It does bring me benefits in some Kaggle competitions.<br>
So I want it to do more, like visualizing pipeline of the dataflow, or even better: configure the dataflow through web interface.</p>

chenglu: <aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://www.redditstatic.com/desktop2x/img/favicon/android-icon-192x192.png" class="site-icon" width="192" height="192">
      <a href="https://www.reddit.com/r/MachineLearning/comments/c53eq1/thoughts_on_visual_development/" target="_blank" rel="nofollow noopener">reddit</a>
  </header>
  <article class="onebox-body">
    <img src="https://external-preview.redd.it/YH8gNap4KoGcFgyFYrNZ86fXmYfRn6pa7uuwIlZkjEE.jpg?auto=webp&amp;amp;s=e62264227377a9581e2e2946169864d130fa3217" class="thumbnail" width="" height="">

<h3><a href="https://www.reddit.com/r/MachineLearning/comments/c53eq1/thoughts_on_visual_development/" target="_blank" rel="nofollow noopener">r/MachineLearning - Thoughts on Visual Development</a></h3>

<p>1 vote and 0 comments so far on Reddit</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


tom: <aside class="quote no-group" data-username="jinfagang" data-post="3" data-topic="48873">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jinfagang/48/9405_2.png" class="avatar"> jinfagang:</div>
<blockquote>
<p>only help for newbies but for masters it’s not needed at all</p>
</blockquote>
</aside>
<p>Well, I always count myself in the <em>still learning</em> group even though I sometimes claim to have a bit of first hand knowledge around PyTorch. That aside, if the vision is that ML/Deep Learning will be as ubiquitous as, say, Excel is today, there is a dire need for tools that are as easy to use as Excel. Python goes a long way in making non-programmers feel at home fast, but it’s not perfect.</p>
<p>Quite likely it  would be beneficial to have something visual that is extensible.<br>
Think about PyTorch itself: You have the ready-made <code>torch.nn.*</code> that you can just compose (with <code>torch.nn.Sequential</code> if you wish) and you can easily roll your own because that’s what we’re there for.</p>
<p>Another learning from this line of thought: A good and general visualization is the first step to a good visual editor.</p>
<p>Best regards</p>
<p>Thomas</p>

chenglu: <aside class="quote no-group" data-username="tom" data-post="9" data-topic="48873">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/48/3162_2.png" class="avatar"> tom:</div>
<blockquote>
<p>Quite likely it would be beneficial to have something visual that is extensible.</p>
</blockquote>
</aside>
<p>Thanks tom, what you said is inspiring. We are developing this kind of tool now.</p>

tom: <p>One other thought: I wonder if there is also inspiration to come from Scratch / <a href="https://developers.google.com/blockly/" rel="nofollow noopener">Blockly</a>.</p>
<p>But then you’re very likely quite ahead of me in thinking this through! I look forward to seeing your results!</p>

chenglu: <p>Yes! Have been thinking about <code>Blockly</code>, but we think it’s more complicated than workflow. The web should be as simple as it could be. So we chose workflow rather than blockly as the frontend interface. <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

tom: <p>Is Workflow a framework? (Sorry, I’m rather clueless.)</p>

chenglu: <p>Nope, it’s a concept, like <a href="https://github.com/features/actions" rel="nofollow noopener">Github Actions</a> or  <a href="https://workflow.is/" rel="nofollow noopener">Apple’s Workflow</a>.</p>

chenglu: <p>Also I’ve been thinking about using TorchScript for the model part. Just saw your tweet about that, really looking forward to see it.</p>