Our proposal introduces a more efficient feedback loop powered by digital labor and enhanced by IBM AutoAI. Instead of relying on experts or engineers, we suggest employing a digital worker to oversee the performance of a pre-trained model. When the model’s accuracy drops, the digital worker will automatically handle tasks such as data collection, re-training, and model evaluation until the accuracy improves. Our proposition introduces a novel approach, shunning the reliance on experts or engineers for the feedback loop process. Instead, we advocate for the integration of a digital worker, entrusted with the responsibility of monitoring the performance of a pre-trained model. With this innovative workforce in place, the digital worker ensures the close monitoring of the model’s efficacy. Should the pre-trained model’s performance show signs of degradation, the digital worker takes immediate action. It seamlessly takes charge of critical tasks such as data collection, re-training, and continuous evaluation of the model, addressing any inefficiencies along the way. This automated cycle ensures that the accuracy of the model is restored to an acceptable level, benefiting the overall feedback loop process. By leveraging the capabilities of IBM AutoAI in conjunction with the digital worker, we are confident that we can streamline the feedback loop, optimizing it for automated document processing. The combined strength of these technologies will result in a more efficient and accurate process, thereby enhancing the overall user experience. The objective we set out to achieve with this implementation is a remarkable enhancement in the user feedback loop for automated document processing. We are aiming for a substantial improvement of 30% to 90%. By automating the various steps involved in monitoring and re-training the model, we anticipate a significant boost in the efficiency of the feedback loop.



Here is an example of a user feedback loop in traditional document processing. An admin user creates a model for document processing by labeling documents and training the model based on these labeled documents. During the testing phase, if the pre-trained model does not perform well, the admin user needs to engage in a time-consuming feedback loop. This loop typically involves experts or engineers examining the test results, identifying problematic documents or fields, and investigating the performance of text extractors. The admin user then makes changes, modifies or creates new extractors in order to improve the model until better results are obtained. This iterative process requires a significant amount of user interaction, leading to reduced efficiency. For many customers, is it possible to automate this user feedback loop and allow admins to focus on simpler tasks?

Our proposal introduces a more efficient feedback loop powered by digital labor and enhanced by IBM AutoAI. Instead of relying on experts or engineers, we suggest employing a digital worker to oversee the performance of a pre-trained model. When the model’s accuracy drops, the digital worker will automatically handle tasks such as data collection, re-training, and model evaluation until the accuracy improves. Our proposition introduces a novel approach, shunning the reliance on experts or engineers for the feedback loop process. Instead, we advocate for the integration of a digital worker, entrusted with the responsibility of monitoring the performance of a pre-trained model. With this innovative workforce in place, the digital worker ensures the close monitoring of the model’s efficacy. Should the pre-trained model’s performance show signs of degradation, the digital worker takes immediate action. It seamlessly takes charge of critical tasks such as data collection, re-training, and continuous evaluation of the model, addressing any inefficiencies along the way. This automated cycle ensures that the accuracy of the model is restored to an acceptable level, benefiting the overall feedback loop process. By leveraging the capabilities of IBM AutoAI in conjunction with the digital worker, we are confident that we can streamline the feedback loop, optimizing it for automated document processing. The combined strength of these technologies will result in a more efficient and accurate process, thereby enhancing the overall user experience. The objective we set out to achieve with this implementation is a remarkable enhancement in the user feedback loop for automated document processing. We are aiming for a substantial improvement of 30% to 90%. By automating the various steps involved in monitoring and re-training the model, we anticipate a significant boost in the efficiency of the feedback loop.

summarize the use cases before / after enhanced user feedback loop.