Question: In fact, from Fig. 7 does it seem that the model just learns to select one best augmentation? Rather than a best combination of all of them.
Evidence: Previous time series contrastive learning methods generate augmentations with either rule of thumb guided by prefabricated human priors or tedious trial-and-errors, which are designed for specific datasets and learning tasks. In this part, we discuss how to adaptively select the optimal augmentations with a meta-learner network based on the proposed information-aware criteria. We can regard its choice of optimal augmentation as a kind of prior selection. We first choose a set of candidate transformations T, such as jittering and time warping. Each candidate transformation t i ∈ T is associated with an important weight p i ∈ (0, 1), inferring the probability of selecting transformation t i . For an instance x, the augmented instance v i through transformation t i can be computed by:
Considering multiple transformations, we pad all v i to be with the same length. Then, the adaptive augmented instance can be achieved by combing candidate ones, v = 1 |T| i v i . To enable the efficient optimization with gradient-based methods, we approximate discrete Bernoulli processes with binary concrete distributions. Specifically, we approximate a i in Eq. () with ∼ Uniform(0, 1)
where σ(•) is the sigmoid function and τ is the temperature controlling the approximation. The rationality of such approximation is given in Appendix A. Moreover, with temperature τ > 0, the gradient ∂v ∂pi is well-defined. Therefore, our meta-network is end-to-end differentiable. Detailed algorithm is shown in Appendix B.
Considering multiple transformations, we pad all vi to be with the same length. Then, the adaptive augmented instance can be achieved by combing candidate ones****We compare performances of variants that each applies a single transformation to generate augmented instances in Table 5. From the table, we know that augmentation with subsequence benefits the most for the Electricity dataset. We visualize the weight updating process of InfoTS in Figure 8, with each line representing the normalized importance score of the corresponding transformation. The weight for subsequence increase with the epoch, showing that InfoTS tends to adopt subsequence as the optimal transformation. Consistency between accuracy performance and weight updating process demonstrates the effectiveness of InfoTS to adaptively select feasible transformations. Besides, as shown in Table 5, InfoTS outperforms the variant that uses subsequence only. This comparison shows that the meta-network learns to consider the combinations, which is better than any (single) candidate augmentation.
Answer: Figure 7 shows the weight updating process of the meta-network on the Electricity dataset. In the initial version, we didn't normalize weights. We have updated the figure with normalized weights in the revised version.  The most important conclusion from the figure is that the optimal augmentation selected by our meta-network is consistent with the one selected with testing accuracy performance. As explained in Section 2.3.2, each candidate augmentation is associated with a weight, and the *"adaptive augmented instance can be achieved by combing candidate ones'' (Page 6)*. In other words, theoretically, the meta-network considers the combination instead of just selecting the best one.  In addition, we have empirical observations in Table 5, InfoTS outperforms the variant that uses subsequence only (the last column). This comparison empirically shows that the meta-network learns to consider the combinations, which is better than any (single) candidate augmentation.