# Research on the Lossless Data Compression System of the Argo Buoy Based on BiLSTM-MHSA-MLP
## 2 Probabilistic Prediction Models
### Probabilistic Prediction Model  
Although the preliminary probability prediction model has achieved a reduction in the compression ratio compared with traditional compression algorithms and traditional deep learning compression algorithms, there has been no significant improvement in the compression time. Therefore, on the basis of the construction and training of the preliminary probability prediction model, an MLP and block_len are introduced to realize the training and block compression of deep-sea profile data. The MLP extracts and transforms features through multiple fully connected layers and nonlinear activation functions to generate the final prediction. This structure can dynamically update the model during the compression process, gradually adapt to the data distribution, and provide more accurate probability predictions, thereby reducing the compression ratio and shortening the compression time. The complete probability prediction model structure is shown in Figure 3.  
First, the single batch of profile data to be compressed is added to the embedding layer, which converts the input data into the corresponding embedding vectors, which can effectively capture the semantic relationship between the data and convert discrete text data into continuous vectors that can be input into the neural network. Moreover, the single batch of profile data to be compressed is input into the trained preliminary probability prediction model to obtain the output result of the Linear1 linear layer in the model and transform its dimension to (batch_size, \(-1\)). The output result of the word embedding layer is concatenated with the result of the preliminary probability prediction model to form a new tensor, which contains rich information from the preliminary probability prediction model, thereby enhancing the feature fusion capability. Figure 2: Preliminary probability prediction model structure. Second, the tensor is input into the MLP network model, which includes three layers, two layers of residual networks and one layer of a linear transformation layer. All three layers use LeakyReLU as the activation function and the slope parameter is 0.005. Because the leaky ReLU function has a small gradient in the negative part, it can solve the problem of neuron death in the ReLU function. Moreover, it is sensitive to the slope parameter and needs to be adjusted. Layer1 consists of one linear layer and one activation function layer; Layers 2 and 3 are both composed of two linear layers and two activation functions. The introduction of multiple fully connected layers can significantly increase the expressiveness of the model. By stacking multiple nonlinear activation functions, the multilayer fully connected network enables the model to better capture complex patterns and nonlinear relationships in the data. The introduction of nonlinearity enables the network to learn more complex function mappings, thereby improving the expressiveness of the model. The introduction of residual networks helps alleviate the vanishing gradient problem. Residual connections allow gradients to flow more easily during back propagation. In addition, the use of residual connections can accelerate the training process of the model. The design of the residual network allows the function learned by each layer to be regarded as a \"correction\" of the input, which allows each layer to learn only the changes in the input instead of the complete representation, thereby simplifying the learning task and promoting faster convergence. Therefore, the combination of multilayer fully connected networks and residual connection networks can improve the stability of the model. Through the above changes, the dimension of the output of the second layer residual network is (batch_size \(\times\) block_len, hdim). Figure 3: Probabilistic prediction model structure. Finally, the linear layer is used to map the learned features to the letter space, generating a tensor in which each element represents the score of a token in the alphabet.