# Development of a Deep Neural Network Model for the
## 3 Model Training  
The goal was to leverage deep learning techniques to accurately predict earthquake parameters based on seismic arrival times and geological features inferred from velocity models. This implementation used PyTorch 2.4 with CUDA support for GPU acceleration, designed to harness the computational power of GPUs through CUDA, enabling faster training and optimization of the model parameters. We imported the torch.nn module from Python 3.12 and built a four-layer neural network model (Figure 3). The first layer was the input layer with 27 neurons, corresponding to the 27 processed arrival times. The activation function used was ReLU. The second layer was a hidden layer with 64 neurons, and the third layer was another hidden layer with 64 neurons, also using the ReLU activation function. The fourth layer was the output layer with 34 neurons, including the source coordinates (X, Y, Z), earthquake time tsts, and velocity model parameters. The activation function for the output layer was also ReLU. We note here, for inversion, we only inverted for a linear depth velocity profile--a horizontal variation of velocities is not accounted for here. Next, we imported the train_test_split function from the sklearn.model_selection module to divide the data into training and test sets using an 80:20 ratio. This means that 80% of the data were allocated to the training set, which was used to train the neural network model. The remaining 20% formed the test set, which served to evaluate the model's performance and assess its generalization capabilities on unseen data. We used TensorFlowDataset and DataLoader from PyTorch to create datasets and data loaders, with a batch size of 128. Figure 3: Neural network architecture used for training in this study. For optimization, we employed the Adam optimizer and utilized the Mean Squared Error (MSELoss) function as our loss metric. The learning rate dynamically ranged from 0.001 to 0.0001 to facilitate efficient convergence during training. Each model is trained for 10,000 iterations and, during each iteration, data undergo one forward and one backward propagation pass, with the loss function's gradient descent optimizing the neural network model parameters. After the iterations are completed, the model is saved for subsequent inversion capability testing and further iterative upgrades. It is important to note that the test data used here are different from the 'true' data that will be used later to validate and assess the performance of our model. The test data here include both arrival times and a velocity model, whereas the 'true' data only have the arrival times without a velocity model. Using a ray-tracing model, we obtained 27 source arrival times (t\({}_{1}\)-t\({}_{27}\)) that count as one set of our training data. We define the minimum value of these 27 arrival times as t\({}_{\text{s}}\). We subtract t\({}_{\text{s}}\) from the arrival times at each station and use these 27 adjusted times as the input data X (Figure 4) for the neural network. The output data Y of the neural network model includes the earthquake source coordinates (X, Y, Z), the earthquake time t\({}_{\text{s}}\), and six velocity parameters for each layer (a total of 180 parameters). We use an iterative training process to train our model with the inclusion of synthetically generated data that are supposed to represent real field data (Figure 5). We first construct our initial training model by inputting 8000 sets of seismic events. This is in accordance with the past engineering experiences that roughly 5000-10,000 events would be recorded per year. Each set in these 8000 data sets has source coordinates that follow a linear pattern, as previously described. These 8000 sets are divided into four subgroups, with each subgroup consisting of 2000 data sets generated from the coordinates (4000, 10,100, 690). Each subgroup employs a different velocity model, with each model described by 180 different parameters (as detailed in the earlier texts).