# Analysis of Machine Learning Techniques for Information Classification in Mobile Applications
## 5 Text Analytics Algorithms on Mobile Devices  
Natural language interface (NLI) helps clients to interact with their computer using high-level language instead of using machine language in the command line interface with the graphical user interface [61]. NLI involves user-computer interaction, helping the computer to understand high-level language, enabling search queries through text or spoken language. Mobile devices present usage challenges for users because they are small and have limited resources and network connection, among other elements [62]. Natural language has been implemented in novel applications such as database queries, question answers, personalisation, etc. NLI focuses on the work of desktop computer systems. NLI on mobile devices expands knowledge theoretically and practically to improve mobile devices. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**PRE-Trained Model** & **Image Input Size (px)** & **Accuracy** & **Parameters** & **Inference Time** & **Size** \\ \hline EfficientNetV2 [54] & \(300\times 300\) & 83\% & 55 M & 57 ms & 220 MB \\ \hline MobileNet [55] & \(224\times 224\) & 70.4\% & 4.3 M & 22.6s & 16 MB \\ \hline MobileNetV2 [55] & \(224\times 224\) & 71.3\% & 3.5 M & 25.9 ms & 14 MB \\ \hline ResNet50 [56] & \(224\times 224\) & 74.9\% & 25.6 M & 58.2 ms & 98 MB \\ \hline VGG16 [57] & \(224\times 224\) & 71.3\% & 138.4 M & 69.5 ms & 528 MB \\ \hline InceptionV3 [58] & \(299\times 299\) & 77.9\% & 23.9 M & 42.2 ms & 92 MB \\ \hline NASNetMobile [59] & \(224\times 224\) & 74.4\% & 5.3 M & 27 ms & 23 MB \\ \hline DenseNEt121 [60] & \(224\times 224\) & 75\% & 8.1 M & 77.1 ms & 33 MB \\ \hline Xception [59] & \(299\times 299\) & 79\% & 22.9 M & 109.4 ms & 88 MB \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Characteristics of the operation of models on mobile devices. Figure 5: EfficientNet-Lite latency (ms) vs Accuracy (top 1) [53]. One area that has attracting attention from researchers is the use of lightweight, pretrained deep learning models to perform text processing on devices, which can allow for the classification of text messages, application messages, notifications, etc., without the need to connect to a remote server to make inferences. However, this task is complex due to hardware limitations such as RAM, storage or battery requirements. Algorithms designed for mobiles must have few training parameters to make the model light enough to be supported. In [63] the authors proposed TinyBERT, a BERT-based model that, to enable its use in resource-constrained devices, makes use of novel methods to distill transformer knowledge both in the pretraining stage and in the task-specific learning stage to be implemented. On the other hand, in [64], MobileBERT, an algorithm based on BERT_Large designed between self-attenuation and feed-forward networks, was proposed. The results obtained by the authors show that MobileBert is 4.3 times smaller and 5.5 times faster than BERT_Base. MobileBERT is a bidirectional transformer based on the BERT model, which is compressed and accelerated using various approaches. Masquerade language modeling (MLM) is effective in predicting masquerade tokens and in NLU in general. MLM is not optimal for text generation, but models trained with a causal language modeling (CLM) goal are better in that regard [63]. The architecture of MobileBERT is shown in the Figure 6.  
DistilBERT is a transformer model that is smaller and faster than BERT. DistilBERT is trained on a corpus previously in self-supervised mode with the base BERT model [65]. DistilBERT was pretrained on three targets: masked language modelling (MLM), loss of cosine embedding and distillation loss. This model is one of the most efficient; in addition to needing few parameters for its training, the space required to store the model makes it ideal for applications on mobile devices. For example, [PERSON] et al. [66], collected data from text posted on social networks and classified how children are harassed by various comments online.