OSError: [Errno 24] Too many open files:

Aarya: <p>Hello,</p>
<p>I have tried to use tensorboard pytorch to plot loss graphs and accuracy.<br>
I have used Summary Writer here.</p>
<p>But, as I run my training loop, I face the following error at the end of first epoch.</p>
<p>Following snippet shows the error:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/6/c66dd3a6eeae1cb9d83e4d7ca568656794629755.png" data-download-href="https://discuss.pytorch.org/uploads/default/c66dd3a6eeae1cb9d83e4d7ca568656794629755" title="summarywritererror"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c66dd3a6eeae1cb9d83e4d7ca568656794629755_2_689x369.png" alt="summarywritererror" data-base62-sha1="sjnQciEY8nliVGED7uBY2NewD1X" width="689" height="369" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c66dd3a6eeae1cb9d83e4d7ca568656794629755_2_689x369.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c66dd3a6eeae1cb9d83e4d7ca568656794629755_2_1033x553.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/6/c66dd3a6eeae1cb9d83e4d7ca568656794629755.png 2x" data-dominant-color="313135"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">summarywritererror</span><span class="informations">1148×614 103 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Following is the python code containing summary writer for reference:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/8/a895f22669f750fd0d8ad983b74f1148d67ff1bb.png" data-download-href="https://discuss.pytorch.org/uploads/default/a895f22669f750fd0d8ad983b74f1148d67ff1bb" title="summarywriter_itr"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a895f22669f750fd0d8ad983b74f1148d67ff1bb_2_690x170.png" alt="summarywriter_itr" data-base62-sha1="o3ntiPKbJdiz3yU1B0BhEvS9E1B" width="690" height="170" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a895f22669f750fd0d8ad983b74f1148d67ff1bb_2_690x170.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a895f22669f750fd0d8ad983b74f1148d67ff1bb_2_1035x255.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/8/a895f22669f750fd0d8ad983b74f1148d67ff1bb.png 2x" data-dominant-color="2E2E2E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">summarywriter_itr</span><span class="informations">1099×272 20.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/f/0fe4b2441e91456d7292e06c0d1006078338fde3.png" data-download-href="https://discuss.pytorch.org/uploads/default/0fe4b2441e91456d7292e06c0d1006078338fde3" title="summarywriter_trainer_2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/f/0fe4b2441e91456d7292e06c0d1006078338fde3_2_689x500.png" alt="summarywriter_trainer_2" data-base62-sha1="2gB8MOsjcLmvUrCPejitqOuL9eP" width="689" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/f/0fe4b2441e91456d7292e06c0d1006078338fde3_2_689x500.png, https://discuss.pytorch.org/uploads/default/original/3X/0/f/0fe4b2441e91456d7292e06c0d1006078338fde3.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/f/0fe4b2441e91456d7292e06c0d1006078338fde3.png 2x" data-dominant-color="2F2F2F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">summarywriter_trainer_2</span><span class="informations">966×701 64.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I also want to plot confusion matrix in tensorboard, please can I have some insights in this as well.</p>
<p>Any help is most appreciated!<br>
Thank you in advance.</p>

ptrblck: <p>Based on the error message your OS runs into the file handler limitation so you would either need to close files (unsure, if Tensorboard opens files without closing them) or could try to increase the file handle limitation.</p>
<p>PS: you can post code snippets or logs by wrapping them into three backticks ```, which makes debugging easier.</p>