## 3 Framework for Automated Earthwork Network Architecture (ENA)
### Module-3: Performance Evaluation  
Module-3: Performance Evaluation in the ENA framework focused on assessing the accuracy, efficiency, and overall effectiveness of the developed models from Module-2. This module involved a series of structured activities that tested and validated the models' performances, ensuring that they could effectively predict and classify earthwork quantities under various conditions. The module began by consolidating all the trained models, including those based on the MLP, LSTM, Transformers, and LLM. These models were then subjected to rigorous training and evaluation processes to gauge their accuracy and reliability in real-world scenarios.  
The initial step within this module was to train the ENA models using the prepared datasets. During this process, the models learned from the structured and pre-processed features extracted from CAD drawings or similar data sources. Each model architecture, whether it was MLP-, LSTM-, Transformer-, or LLM-based, was fine-tuned to optimize its learning capabilities. This stage involved iterating through multiple training epochs, adjusting hyperparameters, and refining the model's weights to ensure optimal learning from the input data.  
Once the training phase was complete, the models proceeded to the test and prediction classification stage. In this phase, each model was evaluated using a designated test dataset that the model had not encountered during training. This testing phase was critical as it measured how well the model generalized to new, unseen data. Various evaluation metrics,such as loss, accuracy, and QTCacc using Equation (9), were used to assess the models' performances. QTCacc, a specific metric developed for evaluating earthwork quantity estimation, was calculated by considering both the area and quantity item classification accuracies. This metric ensured that the models were not only predicting the correct quantities but also correctly classifying each quantity item within the dataset.  
\[\begin{array}{l}QTC_{acc}=(P1+P2)/2\\ P1=A1/\left(A1+A2\right)\\ P2=C1/\left(C1+C2\right)\\ A1=\text{ {Area of Ground True }}\left(\text{m}^{2}\right)\\ A2=\text{ {Area of Ground False }}\left(\text{m}^{2}\right)\\ Q1=\text{{QTC item count of Ground True}}\\ Q2=\text{{QTC item count of Ground False}}\end{array} \tag{9}\]  
During this evaluation phase, it was also essential to monitor the computational efficiency and resource consumption of each model, particularly for LLM-based models that tend to require significantly more computational resources. The LLM-based models, for example, might demonstrate higher accuracy and faster convergence rates but at the cost of increased GPU usage and longer training times. Understanding these trade-offs allowed for a more informed decision on which model architecture was best suited for a specific application or project constraint.  
The final outcome of the Performance Evaluation module was a comprehensive comparison of the models' strengths and limitations. By analyzing their prediction accuracy, classification capabilities, and computational efficiency, this module provided insights into the suitability of each model architecture for automated earthwork quantity estimation. Additionally, it highlighted how advanced models like LLMs and Transformers can outperform traditional architectures when dealing with complex and unstructured datasets. This comprehensive evaluation is crucial in establishing the feasibility and effectiveness of using deep learning models for automated earthwork quantity estimation in the construction industry.