PopEval: A CharacterLevel Approach to EndtoEnd Evaluation Compatible with WordLevel Benchmark Dataset The most prevalent scope of interest for OCR applications used to be scanned documents, but it has now shifted towards the natural scene. Despite the change of times, the existing evaluation methods are still based on the old criteria suited better for the past interests. In this paper, we propose PopEval, a novel evaluation approach for the recent OCR interests. The new and past evaluation algorithms were compared through the results on various datasets and OCR models. Compared to the other evaluation methods, the proposed evaluation algorithm was closer to the humanu0027s qualitative evaluation than other existing methods. Although the evaluation algorithm was devised as a characterlevel approach, the comparative experiment revealed that PopEval is also compatible on existing benchmark datasets annotated at wordlevel. The proposed evaluation algorithm is not only applicable to current endtoend tasks, but also suggests a new direction to redesign the evaluation concept for further OCR researches.

OCRVQA: Visual Question Answering by Reading Text in Images The problem of answering questions about an image is popularly known as visual question answering (or VQA in short). It is a wellestablished problem in computer vision. However, none of the VQA methods currently utilize the text often present in the image. These texts in images provide additional useful cues and facilitate better understanding of the visual content. In this paper, we introduce a novel task of visual question answering by reading text in images, i.e., by optical character recognition or OCR. We refer to this problem as OCRVQA. To facilitate a systematic way of studying this new problem, we introduce a largescale dataset, namely OCRVQA200K. This dataset comprises of 207,572 images of book covers and contains more than 1 million questionanswer pairs about these images. We judiciously combine wellestablished techniques from OCR and VQA domains to present a novel baseline for OCRVQA200K. The experimental results and rigorous analysis demonstrate various challenges present in this dataset leaving ample scope for the future research. We are optimistic that this new task along with compiled dataset will openup many exciting research avenues both for the document image analysis and the VQA communities.