LPGA: LineofSight Parsing with GraphBased Attention for Math Formula Recognition We present a model for recognizing typeset math formula images from connected components or symbols. In our approach, connected components are used to construct a lineofsight (LOS) graph. The graph is used both to reduce the search space for formula structure interpretations, and to guide a classification attention model using separate channels for inputs and their local visual context. For classification, we used visual densities with Random Forests for initial development, and then converted this to a Convolutional Neural Network (CNN) with a second branch to capture context for each input image. Formula structure is extracted as a directed spanning tree from a weighted LOS graph using Edmondsu0027 algorithm. We obtain strong results for formulas without grids or matrices in the InftyCDB2 dataset (90.89% from components, 93.5% from symbols). Using tools from the CROHME handwritten formula recognition competitions, we were able to compile all symbol and structure recognition errors for analysis. Our data and source code are publicly available.

ICDAR 2019 CROHME  TFD: Competition on Recognition of Handwritten Mathematical Expressions and Typeset Formula Detection We summarize the tasks, protocol, and outcome for the 6th Competition on Recognition of Handwritten Mathematical Expressions (CROHME), which includes a new formula detection in document images task ( TFD). For CROHME  TFD 2019, participants chose between two tasks for recognizing handwritten formulas from 1) online stroke data, or 2) images generated from the handwritten strokes. To compare LATEX strings and the labeled directed trees over strokes (label graphs) used in previous CROHMEs, we convert LATEX and strokebased label graphs to label graphs defined over symbols (symbollevel label graphs, or symLG). More than thirty (33) participants registered for the competition, with nineteen (19) teams submitting results. The strongest formula recognition results were produced by the USTCiFLYTEK research team, for both strokebased (81%) and imagebased (77%) input. For the new typeset formula detection task, the Samsung Ru0026D Institute Ukraine (Team 2) obtained a very strong Fscore (93%). System performance has improved since the last CROHME  still, the competition results suggest that recognition of handwritten formulae remains a difficult structural pattern recognition task.