# Prediction of Short-Shot Defects in Injection Molding by Transfer Learning
## 3 Methodology
### Transfer Learning of LT60 Pre-trained Model  
Training a backpropagation neural network requires a large amount of data, and obtaining a substantial volume of actual injection molding data can be challenging. Therefore, in this study, we leveraged a pre-trained LT60 network prediction model, trained with actual injection molding data, to conduct secondary training using a small amount of the LT100 data obtained. This approach effectively reduces the training time and the time required to acquire training data. The process is illustrated in Figure 14.  
We used 2050 points of actual injection data of the LT60 long plate model to train the pre-trained model. In the dataset, the ratio of short-shots to non-short-shots was 4:6. After data preprocessing and correlation analysis, the input for the pre-trained model consisted of 31 feature parameters, and the output was used to indicate whether a short-shot occurred. The data were split into 80% training data and 20% testing data, with an additional 20% taken from the training data for validation. The model architecture consisted of 3 hidden layers: the first layer had 18 neurons, the second layer had 10 neurons, and the third hidden layer had 12 neurons. Dropout regularization was applied after each hidden layer with a dropout rate of 0.3. The model's structure is depicted in Figure 15. Regarding hyperparameter settings, the learning rate was set to 0.001. The activation functions for each layer were as follows: the first layer used Tanh, while the second and third layers used Sigmoid. Adam optimizer was employed for hyperparameter optimization. The batch size for training was set to 32, and the model was trained for 1000 epochs. The hyperparameter settings for the model are outlined in Table 4.  
Figure 14: LT60 pretrained model transfer learning process diagram. Figure 15: LT60 pre-trained model architecture. In the selection of hidden layers for the LT100 transfer learning model, ten different methods were employed to train the transfer learning model, and the results of each method were compared. The methods were as follows:  
* Transfer and freeze all three hidden layers of the pre-trained model;
* Transfer the three hidden layers of the pre-trained model, and freeze the weights of the third hidden layer, then only train the first and second hidden layers of the new model;
* Transfer and freeze the first hidden layer of the pre-trained model, and only train the second and third hidden layers of the new model;
* Transfer and freeze the second hidden layer of the pre-trained model, then only train the first and second hidden layers of the new model. As shown in Figure 16, from the training results, it can be observed that when using the first, fifth, and sixth transfer learning methods, the model's loss in the training set (blue line) and validation set (green line) exhibited a gradual decreasing trend. However, there remained a significant gap between the two, and this gap did not show a tendency to diminish with increasing training iterations. This indicates that the model was likely to experience overfitting as the number of training iterations increased. In the cases of the second, third, fourth, seventh, eighth, and tenth methods, although the model's loss in the training set decreased noticeably, the model's loss in the validation set increased, indicating the occurrence of overfitting in the training process. The training process of the ninth transfer learning method revealed a noticeable decrease in both training and validation set losses, with a simultaneous reduction in the gap between them. This indicates that the model exhibited excellent generalization capabilities. Therefore, we have selected the ninth method for transfer learning in this study. Following the above validation, we selected the ninth hidden layer transfer method to build the transfer learning model. Its architecture consisted of 3 hidden layers.