# Continual Learning for Table Detection in Document Images
## 1 Introduction  
Tables in documents present crucial contents, concisely and compactly. Since tables come in different layouts, even modern optical character-recognition-based methods [1, 2, 3] cannot yield satisfactory performance when extracting information from tables. Therefore, before retrieving tabular information, identifying their position in a document is essential, mainly referred to as _table detection_. Table detection has been a challenging and open problem for the last couple of decades [4, 5]. The fundamental obstacle is the object confusion between tables and other graphical elements in a document that share a similar appearance [6]. Two factors mainly cause this confusion. First, the inherent _high intra-class variance_ due to several distinctive designs and scales in tables. Second, _low inter-class variance_ due to similar appearance to other graphical page objects (figures, algorithms, and formulas). Earlier approaches have addressed such issues by exploiting the external meta-data in documents [7]. Unfortunately, the performance of these methods is usually fouled on scanned document images. Recently, deep-learning-based techniques have given fresh life to the field [8]. We observe a direct association between the advances of table detection and object detection in computer vision [9, 10, 11, 12]. The idea of formulating a table as an object and the document image as a natural scene has produced state-of-the-art results on several publicly available table detection datasets [13, 14]. Although these modern deep-learning-based methods are impressive in many ways, they have some drawbacks. For one, they are data-hungry and only work best when they are exposed to all the possible samples. However, in real-life situations, data scarcity is the norm. Another issue is that artificial neural networks, unlike their natural counterparts (human brain), are not necessarily able to _add_ to their knowledge with the new data thatare fed to them. The problem is _catastrophic forgetting_[15]. This phenomenon is associated with the degradation in the model's performance on prior cases when it is adopted to a new set of data. The primary reason for catastrophic forgetting is that learning necessitates changing the neural network's weights. However, these changes result in forgetting the prior knowledge. Many researchers studied ways to learn from new data over time [16]. In the literature, different names are used to describe these techniques (e.g., _life-long learning, incremental learning, on-line learning_), but here we refer to them as continual learning (CL). The idea of CL is to keep learning from incoming data while preserving the knowledge gained from previous trainings [17]. Figure 1 illustrates the fundamental difference between conventional approaches and our proposed method that leverages continual learning. As it shows, retraining a model on new data results in lower performance. However, a continual learning method preserves the prior knowledge while learning the new knowledge. With the advent of deep learning, we see that enormous datasets are annually introduced for table detection [18; 19]. In order to demonstrate state-of-the-art results, all prior methods require extensive training on the new datasets, hence overriding the previous knowledge. To address these limitations, we propose a novel end-to-end trainable table  
Figure 1: Illustrative sketch of the continual learning usage. After the first training phase (**top** row), conventional methods take the same approach for the next datasets (**middle** row). CL methods can involve previous knowledge in the latest trains by replaying them for the model (**bottom** row). Blue represents true positive, and red denotes false positive. detection method that leverages a continual learning mechanism. The method is robust to the inclusion of novel datasets and exhibits impressive performance on the previously trained data.