Patent Description:
The disclosure herein generally relates to the field of text processing, and, more particularly, to a method and system for generating a data model for text extraction from documents.

Documents contain many images which in turn may contain a lot of information such as but not limited to texts, pictures and graphics. The texts in the images of physical documents is considered as the main data source for analysis task. Text regions can be used to label and train automatic layout learning systems or to detect and locate relevant fields in the document including title, keywords, subheadings, paragraphs and other structured regions (image/table). Segmenting text regions in the image, therefore, is a fundamental step before tasks like text recognition are performed.

State of the art techniques used for document processing and particularly for handling processing of images for data extraction have the disadvantage that they are performance sensitive and computationally complex. They have large computational load and memory footprint, which causes data processing to take long time for inference and in turn makes them not suitable for real-time requirements. D1 (<NPL>") discloses Deploying complex deep learning models on edge devices is challenging because they have substantial compute and memory resource requirements, whereas edge devices' resource budget is limited. To solve this problem, extensive pruning techniques have been proposed for compressing networks. Recent advances based on the Lottery Ticket Hypothesis (LTH) show that iterative model pruning tends to produce smaller and more accurate models. However, LTH research focuses on unstructured pruning, which is hardware-inefficient and difficult to accelerate on hardware platforms. In this paper, we investigate iterative pruning in the context of structured pruning because structurally pruned models map well on commodity hardware. We find that directly applying a structured weight-based pruning technique iteratively, called iterative L1-norm based pruning (ILP), does not produce accurate pruned models. To solve this problem, we propose two activation-based pruning methods, Iterative Activation-based Pruning (IAP) and Adaptive Iterative Activation-based Pruning (AIAP). We observe that, with only <NUM>% accuracy loss, IAP and AIAP achieve <NUM>. 75X and <NUM>$X compression on LeNet-<NUM>, and <NUM>. 25X and <NUM>. 71X compression on ResNet-<NUM>, whereas ILP achieves <NUM>. 77X and <NUM>. 13X, respectively (Abstract). D2 (<NPL>") discloses Incidental scene text detection is a challenging problem because of arbitrary orientation, low resolution, perspective distortion, and variant aspect ratios of text in natural images. In this paper, we present an end-to-end trainable deep model, which can effectively and efficiently locate multi-oriented scene text. Our detector includes a student network and a teacher network, and they inherit complex VGGNet and lightweight PVANet architecture, respectively. While deploying for text detection, the teacher network is used to guide the training process of a student via knowledge distilling so as to maintain the tradeoff between accuracy and efficiency. We have evaluated the proposed detector on three popular benchmarks, and it achieves F-measures of <NUM>%, <NUM>%, and <NUM>% on ICDAR2015 Incidental Scene Text, COCO-Text, and ICDAR2013, respectively, which outperforms the most state-of-the-art methods (Abstract). D3 (<NPL>") discloses to deploy deep neural networks to edge devices with limited computation and storage costs, model compression is necessary for the application of deep learning. Pruning, as a traditional way of model compression, seeks to reduce the parameters of model weights. However, when a deep neural network is pruned, the accuracy of the network will significantly decrease. The traditional way to decrease the accuracy loss is fine-tuning. When over many parameters are pruned, the pruned network's capacity is reduced heavily and cannot recover to high accuracy. In this paper, we apply the knowledge distillation strategy to abate the accuracy loss of pruned models. The original network of the pruned network was used as the teacher network, aiming to transfer the dark knowledge from the original network to the pruned sub-network. We have applied three mainstream knowledge distillation methods: response-based knowledge, feature-based knowledge, and relation-based knowledge (<NPL>), and compare the result to the traditional fine-tuning method with grand-truth labels. Experiments have been done on the CIFAR100 dataset with several deep convolution neural network. Results show that the pruned network recovered by knowledge distillation with its original network performs better accuracy than it recovered by fine-tuning with sample labels. It has also been validated in this paper that the original network as the teacher performs better than differently structured networks with same accuracy as the teacher (Abstract).

Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one embodiment, a processor implemented method of obtaining a data model for text detection is provided. The method includes obtaining a) a training dataset comprising images b) a test dataset comprising images c) a pre-trained base model, d) a plurality of pre-trained weights, and e) an acceptable drop in accuracy with respect to the baseline model, as input. Further, the pre-trained base model is pruned using a Lottery Ticket Hypothesis (LTH) algorithm to generate a LTH pruned data model. Further, the LTH pruned data model is trimmed to obtain a structured pruned data model. Trimming the LTH pruned data model to obtain a structured pruned data model includes iteratively performing the following steps till an accuracy drop of the structured pruned data model is below the acceptable drop in accuracy. In this process, initially a filter sparsity of every filter of each of a plurality of layers of the structured pruned data model obtained after a preliminary pruning of the LTH pruned data model is determined. The determined filter sparsity is compared with a threshold of filter sparsity. Further, all filters for which the determined filter sparsity exceeds the threshold of filter sparsity are discarded, wherein discarding the filters causes structured pruning. Further, the plurality of pre-trained weights are fine-tuned by training the structured pruned data model for a pre-defined number of iterations. Further, the accuracy drop of the structured pruned data model is determined based on the fine-tuned plurality of pre-trained weights. After every iteration if the accuracy drop is still exceeding the acceptable drop in accuracy, a pruning rate affecting rate of pruning of the LTH pruned data model is increased by a pre-defined percentage and for a resulting pruned data model the aforementioned steps are performed. The structured pruned data model is then trained from a teacher model in a Knowledge Distillation algorithm, wherein a resultant data model obtained after training the structured pruned data model forms the data model for text detection.

In another embodiment, discarding the filters includes initially determining number of zeros in every layer of the structured pruned data model. Further, zero percentage in every layer of the structured pruned data model is determined. Further, number of zeros in all layers having the determined zero percentage exceeding a threshold of zero percentage is determined. Further, zero percentage of all filters in all the layers for which the determined number of zeros exceeds a threshold of zeros is determined. Further, all filters for which the determined zero percentage exceeds the threshold of zero percentage are discarded, wherein the filters are discarded by setting corresponding non-zero weights to zero.

In yet another embodiment, a system for obtaining a data model for text detection is provided. The system includes one or more hardware processors, a communication interface, and a memory (<NUM>) storing a plurality of instructions. The plurality of instructions when executed, cause the one or more hardware processors to initially obtain a) a training dataset, b) a test dataset, c) a pre-trained base model, d) a plurality of pre-trained weights, and e) an acceptable drop in accuracy with respect to the baseline model, as input. The system then prunes the pre-trained base model using a Lottery Ticket Hypothesis (LTH) algorithm to generate a LTH pruned data model. The system further trims the LTH pruned data model to obtain a structured pruned data model, by iteratively performing the following steps till an accuracy drop of the structured pruned data model is below the acceptable drop in accuracy. In this process, initially a filter sparsity of every filter of each of a plurality of layers of the structured pruned data model obtained after a preliminary pruning of the LTH pruned data model is determined. The determined filter sparsity is compared with a threshold of filter sparsity. Further, all filters for which the determined filter sparsity exceeds the threshold of filter sparsity are discarded, wherein discarding the filters causes structured pruning. Further, the plurality of pre-trained weights are fine-tuned by training the structured pruned data model for a pre-defined number of iterations. Further, the accuracy drop of the structured pruned data model is determined based on the fine-tuned plurality of pre-trained weights. After every iteration if the accuracy drop is still exceeding the acceptable drop in accuracy, a pruning rate affecting rate of pruning of the LTH pruned data model is increased by a pre-defined percentage and for a resulting pruned data model the aforementioned steps are performed. The structured pruned data model is then trained from a teacher model in a Knowledge Distillation algorithm, wherein a resultant data model obtained after training the structured pruned data model forms the data model for text detection.

In yet another embodiment, the system discards the filters by initially determining number of zeros in every layer of the structured pruned data model. Further, zero percentage in every layer of the structured pruned data model is determined. Further, number of zeros in all layers having the determined zero percentage exceeding a threshold of zero percentage is determined. Further, zero percentage of all filters in all the layers for which the determined number of zeros exceeds a threshold of zeros is determined. Further, all filters for which the determined zero percentage exceeds the threshold of zero percentage are discarded, wherein the filters are discarded by setting corresponding non-zero weights to zero.

In yet another embodiment, a non-transitory computer readable medium for obtaining a data model for text detection is provided. The non-transitory computer readable medium includes a plurality of instructions which when executed, cause one or more hardware processors to perform the following steps to obtain the data model. In this method, a) a training dataset, b) a test dataset, c) a pre-trained base model, d) a plurality of pre-trained weights, and e) an acceptable drop in accuracy with respect to the baseline model, are obtained as input. Further, the pre-trained base model is pruned using a Lottery Ticket Hypothesis (LTH) algorithm to generate a LTH pruned data model. Further, the LTH pruned data model is trimmed to obtain a structured pruned data model. Trimming the LTH pruned data model to obtain a structured pruned data model includes iteratively performing the following steps till an accuracy drop of the structured pruned data model is below the acceptable drop in accuracy. In this process, initially a filter sparsity of every filter of each of a plurality of layers of the structured pruned data model obtained after a preliminary pruning of the LTH pruned data model is determined. The determined filter sparsity is compared with a threshold of filter sparsity. Further, all filters for which the determined filter sparsity exceeds the threshold of filter sparsity are discarded, wherein discarding the filters causes structured pruning. Further, the plurality of pre-trained weights are fine-tuned by training the structured pruned data model for a pre-defined number of iterations. Further, the accuracy drop of the structured pruned data model is determined based on the fine-tuned plurality of pre-trained weights. After every iteration if the accuracy drop is still exceeding the acceptable drop in accuracy, a pruning rate affecting rate of pruning of the LTH pruned data model is increased by a pre-defined percentage and for a resulting pruned data model the aforementioned steps are performed. The structured pruned data model is then trained from a teacher model in a Knowledge Distillation algorithm, wherein a resultant data model obtained after training the structured pruned data model forms the data model for text detection.

In another embodiment, the non-transitory computer readable medium causes discarding of the filters by initially determining number of zeros in every layer of the structured pruned data model. Further, zero percentage in every layer of the structured pruned data model is determined. Further, number of zeros in all layers having the determined zero percentage exceeding a threshold of zero percentage is determined. Further, zero percentage of all filters in all the layers for which the determined number of zeros exceeds a threshold of zeros is determined. Further, all filters for which the determined zero percentage exceeds the threshold of zero percentage are discarded, wherein the filters are discarded by setting corresponding non-zero weights to zero.

<FIG> is a block diagram of a system <NUM> for generating a data model for text processing, according to some embodiments of the present disclosure. The system <NUM> includes or is otherwise in communication with hardware processors <NUM>, at least one memory such as a memory <NUM>, an I/O interface <NUM>. The hardware processors <NUM>, memory <NUM>, and the Input /Output (I/O) interface <NUM> may be coupled by a system bus such as a system bus <NUM> or a similar mechanism. In an embodiment, the hardware processors <NUM> can be one or more hardware processors.

The I/O interface <NUM> may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface <NUM> may include a variety of software and hardware interfaces, for example, interfaces for peripheral device(s), such as a keyboard, a mouse, an external memory, a printer and the like. Further, the I/O interface <NUM> may enable the system <NUM> to communicate with other devices, such as web servers, and external databases.

The I/O interface <NUM> can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, local area network (LAN), cable, etc., and wireless networks, such as Wireless LAN (WLAN), cellular, or satellite. For the purpose, the I/O interface <NUM> may include one or more ports for connecting several computing systems with one another or to another server computer. The I/O interface <NUM> may include one or more ports for connecting several devices to one another or to another server.

The one or more hardware processors <NUM> may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, node machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the one or more hardware processors <NUM> is configured to fetch and execute computer-readable instructions stored in the memory <NUM>.

The memory <NUM> may include any computer-readable medium known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic random-access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, the memory <NUM> includes a plurality of modules <NUM>.

The plurality of modules <NUM> include programs or coded instructions that supplement applications or functions performed by the system <NUM> for executing different steps involved in the process of generating the data model for text extraction from documents, being performed by the system <NUM>. The plurality of modules <NUM>, amongst other things, can include routines, programs, objects, components, and data structures, which performs particular tasks or implement particular abstract data types. The plurality of modules <NUM> may also be used as, signal processor(s), node machine(s), logic circuitries, and/or any other device or component that manipulates signals based on operational instructions. Further, the plurality of modules <NUM> can be used by hardware, by computer-readable instructions executed by the one or more hardware processors <NUM>, or by a combination thereof. The plurality of modules <NUM> can include various sub-modules (not shown). The plurality of modules <NUM> may include computer-readable instructions that supplement applications or functions performed by the system <NUM> for generating the data model for text extraction from documents.

The data repository (or repository) <NUM> may include a plurality of abstracted piece of code for refinement and data that is processed, received, or generated as a result of the execution of the plurality of modules in the module(s) <NUM>.

Although the data repository <NUM> is shown internal to the system <NUM>, it will be noted that, in alternate embodiments, the data repository <NUM> can also be implemented external to the system <NUM>, where the data repository <NUM> may be stored within a database (repository <NUM>) communicatively coupled to the system <NUM>. The data contained within such external database may be periodically updated. For example, new data may be added into the database (not shown in <FIG>) and/or existing data may be modified and/or non-useful data may be deleted from the database. In one example, the data may be stored in an external system, such as a Lightweight Directory Access Protocol (LDAP) directory and a Relational Database Management System (RDBMS). Functions of the components of the system <NUM> are now explained with reference to steps in flow diagrams in <FIG> and <FIG>.

<FIG> is a flow diagram depicting steps of a method <NUM> involved in the process of generating a data model for text processing, using the system of <FIG>, according to some embodiments of the present disclosure.

At step <NUM> of the method <NUM> in the system <NUM> obtains a) a training dataset, b) a test dataset, c) a pre-trained base model, d) a plurality of pre-trained weights, and e) an acceptable drop in accuracy with respect to the baseline model, as input. The training dataset contains one or more documents which may contain one or more images which may contain information/data. The test dataset may contain similar documents and data as in the training data, that may be fed as input to data models for determining efficiency and for various other purposes. For training the pre-trained data model, a variety of documents containing images maybe used. The pre-trained weights maybe used during training of the pre-trained data model, and may be decided based on any standard neural network training approach. The acceptable drop in accuracy with respect to the baseline model indicates extent of accuracy drop permitted or may be tolerated.

Further, at step <NUM> of the method <NUM>, the pre-trained base model is pruned using a Lottery Ticket Hypothesis (LTH) algorithm, to obtain/generate a LTH pruned data model. Further, at step <NUM> of the method <NUM>, the system <NUM> trims the LTH pruned data model to obtain a structured pruned data model. By doing so, a smaller version of the LTH pruned data model is obtained. Various steps involved in the process of trimming the LTH pruned data model by the system <NUM> are depicted in method <NUM> in <FIG>, and are explained hereafter.

The steps in the method <NUM> are iteratively performed till an accuracy drop of the structured pruned data model is below the acceptable drop in accuracy. In this process, at step <NUM> of the method <NUM>, a filter sparsity of every filter of each of a plurality of layers of the LTH pruned data model is determined. The filter sparsity refers to extent/amount of values that are not significant, and are represented as zero. For example, in <FIG>, where there is no sparsity, all cells have been filled, whereas in <FIG> which depicts structural sparsity, empty cells maybe observed. Number of such cells are determined/calculated at step <NUM>. Further, at step <NUM> of the method <NUM>, the determined filter sparsity is compared with a threshold of filter sparsity, and at step <NUM> of the method <NUM>, all filters for which the determined filter sparsity exceeds the threshold of filter sparsity are discarded. Discarding the filters causes structured pruning and a resulting data model after discarding the filters form the structured pruned data model. Various steps involved in the process of discarding the filters are depicted in <FIG> and are explained hereafter.

At step <NUM> of method <NUM> in <FIG>, the system <NUM> determines number of zeros in every layer of the LTH pruned data model. Further, at step <NUM> of the method <NUM>, the system <NUM> determines value of zero percentage in every layer of the LTH pruned data model. The term "zero percentage" refers to percentage of zero values in each layer, and is determined as: <MAT>.

Further, at step <NUM> of the method <NUM>, the system <NUM> determines number of zeros in all layers having the determined zero percentage exceeding a threshold of zero percentage. Further, at step <NUM> of the method <NUM>, the system <NUM> determines zero percentage of all filters in all the layers for which the determined number of zeros exceeds a threshold of zeros. Further, at step <NUM> of the method <NUM>, the system <NUM> discards all filters for which the determined zero percentage exceeds the threshold of zero percentage, wherein the filters are discarded by setting corresponding non-zero weights to zero. For the filters discarded, the system <NUM> set gradient as zero, so that the zeroed weights set while discarding the filters remain as zero even after further training. In an embodiment, value of the threshold of zeros is decided after checking the zero percentage of every layer and by choosing the threshold of zeros as minimum zero percentage among all the layers.

Referring back <FIG>, at step <NUM> of the method <NUM>, the system <NUM> fine-tunes the plurality of pre-trained weights which are obtained as input, by training the structured pruned data model for a pre-defined number of iterations. The pre-defined value of the number of iterations may be changed/adjusted as per requirements. Various steps involved in the process of fine-tuning the plurality of pre-trained weights are depicted in method <NUM> in <FIG>, and are explained hereafter.

At step <NUM> of the method <NUM>, the system <NUM> initializes the structured pruned data model as a model to be trained. While initializing, the system <NUM> may use the plurality of pre-trained weights. Further, at step <NUM> of the method <NUM>, the system <NUM> fine-tunes the structured pruned data model in a plurality of iterations, based on a calculated training loss, to obtain a plurality of trained data models.

To calculate the training loss, the system <NUM> evaluates a model_epoch model and the baseline model to generate results. The system <NUM> further determines Mean Square Error (MSE) loss between the model_epoch model results and baseline model results. This is referred to as MSEloss_results. The MSEloss_results is then compared pixel-wise with the baseline results as: <MAT> and positive_pixel is recorded.

Further, the system <NUM> determines value of sum_loss through positive loss (denoted as "posi_loss") and negative loss (denoted as "nega_loss")
<IMG>.

Referring back to <FIG>, at step <NUM> of the method <NUM>, the system <NUM> determines accuracy in terms of precision and recall, for each of the plurality of trained models, for a test data fed as input. Precision is the fraction of relevant instances among the retrieved instances, while recall is the fraction of relevant instances that were retrieved, and are calculated as: <MAT> <MAT> Where,.

Further, at step <NUM> of the method <NUM>, the system <NUM> retrains a data model that has been identified as having highest accuracy from among the plurality of trained models, to obtain a retrained data model. Further, at step <NUM> of the method <NUM>, the system <NUM> determines accuracy drop of the retrained data model, for the test data fed as input, in comparison with accuracy of a previous model. Further, at step <NUM> of the method <NUM>, the system <NUM> retrains the data model if the accuracy drop is exceeding a threshold of accuracy drop, in iterations, till the accuracy drop is below the threshold of accuracy drop.

Referring back to <FIG>, at step <NUM> of the method <NUM>, the system <NUM> determines the accuracy drop of the structured pruned data model based on the fine-tuned plurality of pre-trained weights. In an embodiment, the system <NUM> determines the accuracy drop of the structured pruned data model with reference to accuracy of a previous version of the structured pruned data model or with reference to accuracy of the baseline model. After every iteration if the accuracy drop is still exceeding the acceptable drop in accuracy, at step <NUM> of the method <NUM>, the system <NUM> increases a pruning rate affecting rate of pruning of the LTH pruned data model by a pre-defined percentage and for a resulting structured pruned data model the aforementioned steps are performed.

Referring back to the method <NUM>, at step <NUM> of the method <NUM>, the system <NUM> trains the structured pruned data model (i.e. latest version for which the accuracy drop is below the acceptable drop in accuracy) is then trained using data from a teacher model in a Knowledge Distillation algorithm, wherein a resultant data model obtained after training the structured pruned data model forms the data model for text detection.

The data model for text detection when deployed for real-time applications, obtains and processes document(s) from which text detection is to be done as input, and outputs the identified text. Results obtained in such an implementation are given in the experimental data section below.

The datasets used during experiments were:.

During the experiments, a native CRAFT model was designed as teacher and compressed CRAFT networks as student with weight initialized by xavier initialization. The training was done on Nvidia A100 GPU with a batch size of <NUM>. The distillation loss, ADAM Optimizer and L2 regularization with a decay of <NUM>-<NUM> is used. Down-scaled models for <NUM> epochs and LTH+channel pruned network for <NUM> epochs were trained.

For inference, the model was put in evaluation mode and tested it on IC15 and IC19 datasets. As part of pre-processing, test images were scaled to <NUM>×<NUM>×<NUM> followed by image normalization using mean and variance. The model produced output sized (h/<NUM>,w/<NUM>,<NUM>) i. e (<NUM>,<NUM>,<NUM>), region and affinity feature maps. In the post-processing step, the output was converted to heat maps and polygons created over predicted textual regions. The predicted text boxes were characterized as correctly predicted by calculating its overlap (IoU threshold = <NUM>) with boxes from ground truth. From the predicted and ground truth text boxes, accuracy (precision and recall) of model was estimated.

Accuracy and compression data for six downscaled and three LTH+channel pruned models are given in Table I. The student architectures were compressed by <NUM>-<NUM>× with accuracy drop in range <NUM>%-<NUM>% for IC15 and <NUM>%-<NUM>% for IC19 dataset. As expected, precision and recall for extremely compressed models (C4, C2 and C1) faced large drop (i10%) in accuracy. C32-C8 architectures retained accuracy within <NUM>% drop but C32/C16 were only moderately compressed and consequently would not yield significant performance gains for the given accuracy drop.

Among the down-scaled architectures, C8 model exhibited high compression with <NUM>% accuracy drop. The data model generated by the system <NUM> suffered lower accuracy drop despite high compression. For similar compression, the data model generated by the system <NUM> have higher accuracy as compared to down-scaled models. For instance, <NUM>× compressed model, C8 has lower accuracy as compared to C18Pr model that is <NUM>× compressed. Additionally, C6Pr offered the same accuracy as C8 model but offered higher (<NUM>×) compression.

The precision of computation denotes the number of bits and datatype used in calculations. Lower precision compute offers higher computational power (in terms of GOP/S) which translated to performance improvements. For instance, V100 GPU supports FP32 and FP16 computations but the latter has <NUM>× the computational power [<NUM>]. However, reducing the precision affects the prediction accuracy since the number of bits used in computation reduces. Effect on accuracy due to compute precision reduction was analyzed for native and compressed CRAFT models on V100 GPU, U280 FPGA DPU and Xeon CPU. Among the selected hardware platforms, all supported INT8 precision. In addition, Xeon supports FP32 and V100 supports both FP32 and FP16. The CRAFT and its compressed models were compiled for FPGA using VITISAI quantizer. TensorRT (maps computation to tensor cores) and Pytorch are used for quantizing models on GPU and CPU, respectively. F1-score was chosen as the accuracy metric to capture the essence of both Precision and Recall. It takes a value between [<NUM>,<NUM>] where higher score denotes high precision and recall. The accuracy variation with compute precision on IC15 dataset is shown in <FIG>. It was observed that all CRAFT models: native and compressed, suffered <NUM>-<NUM>% accuracy loss from their respective FP32 accuracy due to reduced precision. However, with respect to native CRAFT, the accuracy loss of compressed models is upto <NUM>%. Among the compressed models, ChPr+LTH models suffered lower accuracy loss (≤ <NUM>%) than the down sampled models. This indicates that ChPr+LTH models perform better on accuracy than downsampled models after reducing precision.

Another observation is on the compilers for target hardware platforms. The VITIS AI quantizer and GPU compiler are able to compress models from FP32 to INT8 with an accuracy drop of <NUM>-<NUM>%. In case of VITIS AI quantizer, a calibration step is carried out that uses a small portion of test data to manipulate the quantized weights so that the accuracy drop can be minimized. Additionally, the results suggest that smaller models (C6Pr, C4) suffer lower accuracy loss with precision variation. This can be attributed to the lesser parameters and computational load of small models.

The native and compressed CRAFT models were tested for inference time on GPU (A100, V100), FPGA (U280 DPU) and Xeon CPU. The performance of CRAFT model was tested for all supported precisions on target hardware platforms. In addition to TensorRT, Apex library is used to compile models for A100. The Apex library introduces <NUM>:<NUM> sparsity into models. <NUM>:<NUM> sparse models were supported on A100 GPU owing to its special hardware. The maximum throughput achieved by CRAFT models was on A100 GPU with INT8 precision in Apex and TensorRT compile (see Table IV). The native CRAFT model achieved the highest throughput of <NUM> fps on A100 with INT8 precision and Apex compilation. This is 2fps higher than compilation on TensorRT for same hardware and precision. However, the benchmark presented A100 documentation suggested the speedup with Apex to be <NUM>× as compared to other compilation. From a detailed layer-wise analysis, it was observed that the Apex speed-up is limited to convolution layers and other layers like batch normalization, maxpool etc. are not accelerated. In the CRAFT model, the batch normalization is the bottleneck layer and even though convolution layers are accelerated, apex compilation did not yield considerable speed-up over TensorRT. The throughput achieved on all target hardware is shown in <FIG> for their respective best batch size. The best batch size for A100, V100 and FPGA DPU are <NUM>, <NUM> and <NUM>, respectively. The CPU throughput ranges between <NUM>-<NUM> fps. The INT8 C6Pr model gets <NUM> × speed-up w. FP32 implementation of native CRAFT. Additionally, C8 and C4 models showed <NUM>-<NUM>× significant performance improvement on CPU. The compressed models offered <NUM>-<NUM>× speed-up over native CRAFT and GPU implementations providing highest throughput. The FPGA DPU offers throughput between <NUM>-<NUM> fps. However, this observation alone did not rule out the usability of FPGA DPUs as preferable neural processors.

From the performance perspective, INT8 implementation performed better than FP32 because the computational power of hardware is higher at lower precision. However, while selecting the hardware platform and precision for a particular model, the effect on accuracy should also be considered. <NUM>) Effect of Batch-size on throughput: For throughput measurement experiments, average time to run n calls to a batch of b images was measured. It was observed that it took large time to run the first batch whereas the subsequent batches were able to finish in three order lesser time. This behaviour was seen dominantly in GPUs for all CRAFT variants. The large time (in order of sec) taken during first run is spent in transferring the model parameters, loading library engines (TensorRT, Pytorch, Apex etc.) and prepare the GPU for upcoming workload. This was defined as a priming time. The throughput data did not include GPU priming time. However, in a real-deployment scenario priming time cannot be neglected especially since it ranges in few seconds. Variation of throughput with batch size for C6Pr INT8 model on IC15 dataset in <FIG>. It was observed that DPU have higher throughput as compared to GPUs for batch size upto <NUM> images (indicated as DPUGPU crossover). For batch size upto <NUM> (indicated as A100-V100 crossover1) images, A100 and V100 had similar throughput beyond which A100 offers higher throughput than V100. The two GPUs performed similar for batch size upto <NUM> (indicated as A100-V100 crossover2) if priming time is accounted for in throughput. For large batch sizes, the priming time gets amortized over large number of images and consequently the throughput improves consistently. This observation indicated that hardware platform with best throughput changes with batch size, which further depends on the workload and nature of application.

Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims.

The embodiments of present disclosure herein address unresolved problem of document processing and text detection using low memory footprint and non-process intensive approach. The embodiment, thus provides a method and system for generating a data model for text identification from documents. Moreover, the embodiments herein further provides a mechanism of pruning a baseline data model which involves discarding filters that have sparsity exceeding a threshold of sparsity.

Claim 1:
A processor implemented method (<NUM>) of obtaining a data model for text detection from an input image, comprising:
obtaining (<NUM>), via one or more hardware processors, a) a training dataset comprising images, b) a test dataset comprising images, c) a pre-trained base model, d) a plurality of pre-trained weights, and e) an acceptable drop in accuracy with respect to the baseline model, as input, characterized in that:
pruning (<NUM>), via the one or more hardware processors, the pre-trained base model using a Lottery Ticket Hypothesis (LTH) algorithm to generate a LTH pruned data model;
trimming (<NUM>), via the one or more hardware processors, the LTH pruned data model to obtain a structured pruned data model, comprising iteratively performing till an accuracy drop of the structured pruned data model is below the acceptable drop in accuracy:
determining (<NUM>) a filter sparsity of every filter of each of a plurality of layers of the LTH pruned data model;
comparing (<NUM>) the determined filter sparsity with a threshold of filter sparsity;
discarding (<NUM>) all filters for which the determined filter sparsity exceeds the threshold of filter sparsity, wherein discarding the filters causes structured pruning and a resulting data model after discarding the filters form the structured pruned data model;
fine-tuning (<NUM>) the plurality of pre-trained weights by training the structured pruned data model for a pre-defined number of iterations;
determining (<NUM>) the accuracy drop of the structured pruned data model based on the fine-tuned plurality of pre-trained weights; and
increasing (<NUM>) a pruning rate affecting rate of the preliminary pruning of the LTH pruned data model, by a pre-defined percentage; and
training (<NUM>), via the one or more hardware processors, the structured pruned data model from a teacher model in a Knowledge Distillation algorithm, wherein a resultant data model obtained after training the structured pruned data model forms the data model for the text detection.