## 5 Discussion  
In this study, key differences between the performance of the MLP, LSTM, and LLM-based ENA models for earthwork quantity classification are highlighted. This is different from previous studies [33; 35] that train, classify, and predict the patterns through text sequences only in the AEC domain. Firstly, the MLP and LSTM-based models utilize one-hot encoding for training datasets, which, while effective, is limited in capturing complex relationships within the data. In contrast, the LLM-based ENA model employs feature tokens within the input sequence, allowing it to better understand and process unstructured data, such as CAD entities, by transforming them into a structured sequence of tokens. On the other hand, existing studies [28; 39] using sequence modeling such as LSTM only implemented cases where the learning data were already expressed in the form  
Figure 10: The ENA model’s Earthwork Quantity Takeoff Classification estimation results (\(\propto\) ENA model ID, y: stations. For the color code, refer to the earthwork item color code defined in Figure 9). of time series dataset, so the research that considers such an informal dataset in AEC is not discussed. Secondly, the LLM-based model excels at training sequence relationships within input tokens, unlike the MLP and LSTM models, which treat input variables more like regression curve fitting. This enables the LLM model to handle a broader range of data types, such as CAD drawings and images, and provides faster convergence. However, the LLM requires considerably more GPU resources. For instance, by epoch 10, the loss for the MLP-based M1.1 model was 0.1685, whereas the LLM-based model had already achieved a much lower loss. This faster convergence highlights the LLM's efficiency in managing complex datasets. Thirdly, MLP and LSTM models are effective alternatives in environments where GPU resources are limited. Despite their more straightforward architectures, these models achieved high accuracy, between 0.94 and 0.95, with low loss values (0.08 to 0.13), and maintained smaller model sizes (84 KB to 3312 KB). This makes them a practical choice when computational resources are scarce, though their ability to handle more complex data is comparatively limited. Fourthly, the LLM-based ENA model demonstrated superior performance in handling unstructured vector data, such as that used in earthwork tasks. Its training accuracy reached 0.95, with an exceptional QTCacc of 97.17%. This shows the LLM model's ability to encode and process diverse input features, making it highly versatile for various construction-related applications. In addition, there was also the previous study using Transformers [29], but since the training dataset was developed based on virtual scenarios rather than actual data in AEC, there was no realistic discussion. However, adopting LLM-based models requires careful consideration. The first critical factor is the development of an effective strategy for preparing training data. Datasets like CAD drawings must be tokenized into sequences, and the way these sequences are encoded can significantly impact the model's training performance and efficiency. Optimizing the encoding process is essential to fully leverage the LLM's capabilities. Second, while Transformer-based models reduce the need for extensive feature engineering, encoding training data for the LLM-based ENA model still requires considerable effort to ensure that the token sequences are well structured. Additionally, due to the LLM's high demand for GPU resources, it is crucial to optimize the encoding process to minimize computational costs. Lastly, training Transformer models with small datasets often leads to suboptimal performance, as demonstrated by the M3.1 model, which only achieved an accuracy of 0.774. To improve performance when dealing with limited data, it is advisable to use pre-trained models or employ data augmentation techniques such as Retrieval-Augmented Generation (RAG).