I need advice for a OCR project

Miguel_Campos: <p>Hello, I am working in a important OCR ( Optical Character Recognition) project which I need to read labels in a industry field. Here and example:</p>
<p>This is what we have</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/5/b54b3caafeaa79e534d95232513be14f9514f080.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/b54b3caafeaa79e534d95232513be14f9514f080" title="dsa"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/5/b54b3caafeaa79e534d95232513be14f9514f080_2_690x442.jpeg" alt="dsa" data-base62-sha1="pRNCeo77JeAwHTbUScfdX3iTxE4" width="690" height="442" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/5/b54b3caafeaa79e534d95232513be14f9514f080_2_690x442.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/b/5/b54b3caafeaa79e534d95232513be14f9514f080_2_1035x663.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/5/b54b3caafeaa79e534d95232513be14f9514f080_2_1380x884.jpeg 2x" data-dominant-color="575656"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">dsa</span><span class="informations">2808×1800 1.76 MB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So to read the labels in this package, what we need to do is, first locate the region of interest, then locate where have texts and finally to apply OCR. So the steps and results could be described as follow</p>
<p><strong>1. ROI detect</strong></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/a/0a3060036b27605459cf8c5c79b7353cf2e5d875.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/0a3060036b27605459cf8c5c79b7353cf2e5d875" title="00001_CAMERA_04_2d (1)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a3060036b27605459cf8c5c79b7353cf2e5d875_2_690x442.jpeg" alt="00001_CAMERA_04_2d (1)" data-base62-sha1="1s8pTVEUehf9mKhmz4MkHNizDdb" width="690" height="442" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a3060036b27605459cf8c5c79b7353cf2e5d875_2_690x442.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a3060036b27605459cf8c5c79b7353cf2e5d875_2_1035x663.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/a/0a3060036b27605459cf8c5c79b7353cf2e5d875_2_1380x884.jpeg 2x" data-dominant-color="525252"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">00001_CAMERA_04_2d (1)</span><span class="informations">2808×1800 1.69 MB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/a/5a3f369e9d69c2a26e2fe851c178bb5b098b97dd.png" alt="1111" data-base62-sha1="cSmpbWXTIzAfoUk75k3GFASe7Fz" width="327" height="466"></p>
<p><strong>2. Text Location</strong></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/e/5ed43acbf425569065c28eadd7d6e7e7af3b3ae2.png" alt="1111" data-base62-sha1="dwTA5ZgxcH7MqBPGaKz0BwHAnvQ" width="327" height="466"></p>
<p><strong>3. Text Recognition/OCR</strong></p>
<p>I have been working in the <strong>Text Recognition</strong> (3 step) with good results using this pytorch network</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/clovaai/deep-text-recognition-benchmark">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/clovaai/deep-text-recognition-benchmark" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/263d937a0a171183dbcd0a55a227db8845f1f3fb0882dcbfbaca15d0576b011c/clovaai/deep-text-recognition-benchmark" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/clovaai/deep-text-recognition-benchmark" target="_blank" rel="noopener nofollow ugc">GitHub - clovaai/deep-text-recognition-benchmark: Text recognition (optical...</a></h3>

  <p>Text recognition (optical character recognition) with deep learning methods. - GitHub - clovaai/deep-text-recognition-benchmark: Text recognition (optical character recognition) with deep learning ...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Okay, this is a good point of start for all of you guys ;D</p>
<p><strong>The main question of this topic is, what is your advice for steps 1 and 2?</strong></p>
<p>I though to train a Faster RCNN (ResNet+FPN) to detect barcodes. I could generate white and empty synthetic labels with just a barcode (I found some webs with barcode fonts to dowload) but i am open to your proposes <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Dwight_Foster: <p>That looks pretty good. One recommendation I could make is you might want to check out the detectron2 model. It is the new state of the art object detection model and is in pytorch. You can find the github <a href="https://github.com/facebookresearch/detectron2" rel="noopener nofollow ugc">here</a></p>

Miguel_Campos: <p><a class="mention" href="/u/dwight_foster">@Dwight_Foster</a> yeah thank you I Knew detectron2 :). I had planned use it for faster.</p>
<p>Another option could be a segmentation with mask rcnn…</p>

Dwight_Foster: <p>Yes you could do that. I think detectron2 can do masks too. You could also try the faster rcnn with a mobilenet backbone.</p>