# A Compression-Based Toolkit for Modelling and Processing Natural Language Text
## 4 Sample Applications Provided by the Toolkit
### Implementation Details, Memory Usage and Execution Speeds
These models store the trie data structure that records all the contexts in the training text and their predictions (i.e., using the frequencies of symbols that follow the context) up to the maximum order of the model. So an order 6 model, for example, will contain information about all contexts of lengths 6 down to 0 plus their predictions. The symbol frequencies stored in the trie are used by the toolkit's libraries for estimating probabilities and for arithmetic coding purposes. In order to reduce the size of the dynamic model as it is being updated, each dynamic model also maintains pointers back into a copy of the input text when a context becomes unique and this is stored in conjunction with the model's trie data structure. Also included in the table are two further texts added for illustration purposes to highlight aspects of the static and dynamic model mechanisms provided by the toolkit. The first, indicated by '...x 2' in the table, is for when a dynamic model was first created from the [PERSON] corpus and written out, then \begin{table}
\begin{tabular}{l r r r r r r r} \hline \hline
**Text** & \begin{tabular}{c} **Size** \\ **(Bytes)** \\ \end{tabular} & \begin{tabular}{c} **Order** \\ \end{tabular} & \begin{tabular}{c} **Static Size** \\ **(Bytes)** \\ \end{tabular} & \begin{tabular}{c} **Avg. Time** \\ **(seconds)** \\ \end{tabular} & \begin{tabular}{c} **Dynamic Size** \\ **(Bytes)** \\ \end{tabular} & \
\begin{tabular}{c} **Avg. Time** \\ **(seconds)** \\ \end{tabular} &
\begin{tabular}{c} **Comp. Input** \\ **(\%)** \\ \end{tabular} \\ \hline LOB corpus & 5,877,271 & 6 & 23,114,666 & 7.859 & 41,067,518 & 8.279 & 16.23 \\ LOB corpus & 5,877,271 & 5 & 128,515,26 & 6.811 & 22,667,634 & 7.177 & 8.35 \\ LOB corpus & 5,877,271 & 4 & 5,766,170 & 6.101 & 10,141,414 & 6.476 & 3.39 \\ LOB corpus & 5,877,271 & 3 & 1,920,470 & 5.996 & 3,350,666 & 6.023 & 0.96 \\ Brown corpus & 5,998,525 & 6 & 24,168,398 & 7.734 & 42,957,274 & 8.343 & 16.73 \\ Brown corpus & 5,998,525 & 5 & 13,357,088 & 7.512 & 23,628,424 & 7.892 & 8.54 \\ Brown corpus & 5,998,525 & 4 & 5,979,854 & 6.061 & 10,515,322 & 6.517 & 3.43 \\ Brown corpus & 5,998,525 & 3 & 1,993,502 & 6.170 & 3,477,030 & 6.468 & 0.97 \\ Brown corpus \(\times\) 2 & 11,997,050 & 5 & 18,679,790 & 8.050 & 31,132,970 & 8.461 & 0.00 \\ Brown corpus \(\times\) 3 & 17,995,575 & 5 & 18,679,790 & 8.544 & 31,132,970 & 8.815 & 0.00 \\ English Bible & 4,138,511 & 5 & 4,523,104 & 2.927 & 7,846,200 & 3.035 & 2.79 \\ Latin Bible & 3,399,860 & 5 & 4,229,580 & 2.454 & 7,324,732 & 2.520 & 3.04 \\ Tipster WSJ & 132,193,779 & 5 & 53,915,630 & 233.626 & 95,069,366 & 223.100 & 1.48 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Sizes (in bytes) of text and models, average training execution speeds (in seconds) and dynamic model compressed input percentages for various sample texts. the same [PERSON] corpus text was added again after the model was reloaded by the **train** tool, and then eventually the model was written out to a different model file (either as a static or dynamic model). The second, indicated by '\(\ldots\times 3\)' in the table, is for the case when the [PERSON] corpus was used three times to train the final model. The table also provides average execution times to build the models across 10 runs (on the same MacBook Pro used for the experiments reported in Table 4) in columns 5 and 7 for static and dynamic models respectively. In general, the time for building a static model takes roughly 1 MB per second and is on average slightly quicker than for building a dynamic model. However, the time required to build models will depend on the complexity of the model and the length of the text with the models trained on repeated Brown corpus text, for example, being more complex and therefore slower to build, and the order 5 model for the much longer Tipster WSJ text having a much faster build rate of 1.85 MB per second. For the model sizes, the results show that the sizes of the model grow substantially with order, but the rate of growth in model size reduces with higher orders.