## 4 Performance Evaluation Through a Case Study
### Prototype Development  
To compare the performance between the ENA models, an ENA prototype was developed (Figure 7). The models were derived from PyTorch Module, and forward () function of M1, M2, M3, and M4's ENA models were implemented. The architecture used in each model is the same as the formula described above. Autocad Python library was used to collect each entity's data in the cross-section. ENA_quantity_item, which expresses each earthwork item, kept the label and polyline. After generating the connectivity of the entity stored in the cross-section by layer, the features were calculated and they were stored in the earthwork feature. Training features were converted into token sequences via one-hot encoding or the EAN_feature_tokenizer according to each model type. Transformers connected the Transformer Encoder output to the MLP to classify earthwork. The LLM used pre-trained BERT, and the same data format was used as a Transformers-based ENA model with a token sequence that was converted for model fine-tuning.  
After the ENA's M1-4 models were developed in PyTorch (2.4), they were trained using hyperparameters like Table 1. The 524 GPU used was NVIDIA RTX 3080 with 8 GB RAM (Nvidia, Santa Clara, CA, USA). The table provides a comprehensive overview of the hyperparameters used to train various models within the ENA. These models were categorized into different architectures, including the MLP, LSTM networks, Transformers, and LLM, each with its own configuration and training setup.  
For batch size, all MLP and LSTM models were trained using a batch size of 32, while the batch size for Transformers-based models varied across 32, 64, and 128, reflecting the increased computational demands of more complex architectures. In terms of epochs, MLP and LSTM models were trained for 150 epochs, ensuring adequate learning for these simpler models. However, for the more complex Transformers and LLMs, training was  
Figure 6: CAD drawings (a portion of the cross-sections) for the case study.  
Figure 7: ENA prototype architecture (UML).  
extended to 300 epochs to allow the models sufficient time to learn from the data and converge on accurate predictions. The layer architecture column describes the specific configurations for each model.  
The MLP models used fully connected layers with various configurations, such as [14]-[128; 64; 32]-[10], indicating the number of neurons in each layer. The LSTM models incorporated stacked LSTM layers with configurations like [14]-LSTM[128]-[10]. Transformers and LLMs, being more advanced, employed an embedding layer (EMB) followed by encoder layers (ENC) to process input data more effectively. Regarding normalization and activation functions, the MLP models rely on ReLU activations, batch normalization, and dropout for regularization. The LSTM models primarily use dropout to prevent overfitting, while Transformer models utilize multi-head attention, layer normalization, and dropout to enhance performance and ensure stability during training.  
The learning rate was set at 0.001 for the MLP and LSTM models, which is a typical choice for these simpler architectures. For the more complex Transformers and LLMs, the learning rate was reduced to 1.00 \(\times\) 10\({}^{-5}\) to facilitate finer adjustments to the model parameters during training, avoiding overshooting and ensuring smooth convergence. Lastly, pre-trained models were used for the Transformers and LLM architectures. Specifically, these models leveraged BERT (base-uncased) embeddings to enhance their performance in processing text-based data for earthwork quantity estimation tasks. The use of pre-trained models allowed the network to start with more informed weights, reducing the amount of data needed for training and improving accuracy.