## 3 Framework for Automated Earthwork Network Architecture (ENA)
### Module-2: ENA Model Development
Each component in the figure represents a step in the framework's systematic approach to structuring and processing data, followed by developing models based on the type of input data and desired outcomes. Once the datasets were created, they were utilized to train four types of ENA models, each leveraging a unique deep learning architecture. The M1 model processed fixed-position data and was tailored for simpler datasets. The M2 model employed LSTM networks to learn temporal dependencies in sequential data, making it effective for applications that required time-based pattern recognition. The M3 model applied Transformer networks, which were designed to handle relationships between elements in large datasets, making them ideal for modeling complex dependencies. Finally, the M4 model integrated pre-trained BERT models, enabling it to process contextual information within sequence data. By leveraging BERT's pre-trained embeddings, the LLM-based model could learn from unstructured inputs with greater accuracy and efficiency. The workflow culminated in the integration and training of these ENA models. Each model was fine-tuned to learn from its respective dataset, optimizing its performance based on the specific requirements of earthwork quantity estimation. The pre-trained BERT model, incorporated into the LLM-based ENA, further enhanced the model's ability to process and learn from complex datasets by providing contextual embeddings that added depth to the learning process. The following equations define the ENA models M1 through M4. M1. The MLP-based ENA was designed as (5) in consideration of training data structure and model tuning. \[ \begin{array}{l}\mathit{EAN}_{MLP}=\{front\;fc,\;MLP\;module,\;classification\;fc\}\\ \mathit{fc}=\mathit{fully\;connection}\\ \mathit{input}=\mathit{input\;size}\\ \mathit{hidden}=\mathit{hidden\;layers}^{\prime}\mathit{s\;units\;size\;list}\\ \mathit{output}=\mathit{output\;size}\\ \mathit{ReLU}(x)=\max\;(0,\;x)\\ \mathit{BatchNorm}(x)=\left(x-E[x]\right)/\sqrt{\left(\mathit{Var}[x]+\epsilon \right)}\times\gamma+\beta\\ \gamma,\;\beta=\mathit{learnable\;parameter\;vectors\;of\;input\;size}\\ \hat{\gamma}=\mathrm{output\;of\;hidden\;layer}\\ \hat{\gamma}=\mathrm{predicted\;output}\\ \end{array}  \tag{5}\]  
M2. The LSTM-based ENA was designed as Equation (6) [40]. \[ \begin{array}{l}\mathit{EAN}_{LSTM}=\{front\;LSTMs,\;MLP\;module,\; classification\;fc\}\\ \mathit{LSTM}=\{LSTM*\}\\ \mathit{cell}=\{i_{t},f_{t},o_{t}\}\\ \end{array}  \tag{6}\]M3. The Transformers-based ENA was defined as Equation (7) [41] to train the ENA dataset. \[\begin{array}{l}\begin{array}{l} \begin{array}{l}EAN_{TR}=\{front\ fc,\ TR_{ encoder},\ classification\ fc\}\\ here,\\ \end{array} \\ \begin{array}{l}TR_{enc}=transformer_{encoder}\Big{\{}feature,\ TR_{layer}* \Big{\}}\\ \begin{array}{l}EAN_{enc\_layer}=\{Embedding,\ Multihead\ Attention,\ Add\&Norm,\ MLP\}\\ \begin{array}{l}Multihead\ Attention=\{Attention*\}\\ \end{array}\\ \begin{array}{l}Attention=softmax\big{(}QK^{T}/\sqrt{d_{K}}\big{)}\cdot V\\ \begin{array}{l}Add\&Norm=\{addition,\ normalization\}\\ \end{array}\\ \begin{array}{l}Q=query\ vector,\ K=key\ vector,\ V=value\ vector\\ \end{array}\\ \end{array}\end{array}\end{array}  \tag{7}\]  
M4. The LLM-based ENA was defined as Equation (7). The model ran the base LLM after embedding the data to train the ENA dataset. The input sequence was converted to embedding vectors after tokenization. \[\begin{array}{l} \begin{array}{l}EAN_{LLM}=\big{\{}LLM_{pretrain},\ fc,\ softmax\big{\}}\\ \begin{array}{l}here,\\ \text{fc}=\text{fully connected layer}\\ \end{array}\\ \begin{array}{l}softmax(x_{i})=e^{x_{i}}/sum(e^{x_{i}})\\ e^{x_{i}}=exponent\ of\ x_{i}\\ e^{x_{j}}=exponent\ of\ x_{j}\\ sum=summation\ function\end{array}\\ \end{array}  \tag{8}\]  
The LLM-based model training calculated the mask vector required for training in advance according to the length of the input sequence and was delivered to the model with the dataset. As a result, \(EAN_{LLM}\) was fine-tuned to suit the classification of earthwork.