Paper: Disentangling Time Series Representations via Contrastive Independence-of-Support on l-Variational Inference (ICLR-2024)

Reviewer comment (Experiments):
Please discuss how sensitive the method is to hyperparameter selection.

Author response:
Dear Reviewer cDdv,

We trust this message finds you well. Your feedback has played a crucial role in improving the analysis of how sensitive the method is to hyperparameters, and we sincerely appreciate your time and attention. 

We've carefully incorporated your suggestions in both the main paper and appendix, aiming to meet your expectations with this revised version. Thank you for your valuable input, and we appreciate your time and consideration.

Best regards,

Authors,

Reviewer comment (Writing):
The paper needs some modifications to make it easier to read (some suggestions given in the Questions).

Author response:
We appreciate the feedback regarding the readability and abstract nature of the experimental results and application description. We have addressed these concerns by providing additional details and clarification in the revised version (which will be updated during this discussion). Specific modifications have been made based on the suggestions provided in the Questions section.

Reviewer comment (Evaluation):
The experimental results are very abstract (some suggestions in Questions part).

Author response:
Experimental results section now includes specific examples illustrating how $\textbf{x}$ and $\textbf{y}$ values (see section 5, other results in the appendix) for a time window are displayed for each experiment, addressing concerns about abstractness we add the figure of mixed signal input and unmixed results (prediction).

Reviewer comment (Presentation):
The application worth more explanation, the description lacks either an illustration or it is abstract.

Author response:
We appreciate the feedback regarding the readability and abstract nature of the experimental results and application description. We have addressed these concerns by providing additional details and clarification in the revised version (which will be updated during this discussion). Specific modifications have been made based on the suggestions provided in the Questions section.

Reviewer comment (Theory):
Using cosine similarity instead of HFS needs more elaboration.

Author response:
We have provided a more elaborate more explanation for Hausdaurff Factorization support (HFS) in the revised version.  
We favor Cosine Similarity over alternative distance measures based on specific considerations within our context. Our choice is motivated by the computational efficiency of Cosine Similarity compared to Earth Mover's Distance (EMD), as outlined in our study detailed in Appendix C. Notably, we observed that Cosine Similarity exhibits scale-invariance, focusing on direction rather than being affected by the magnitude of the latent. In contrast, EMD incorporates transportation costs between distributions, introducing a sensitivity to scale.

Reviewer comment (Presentation):
Section 3.2 would benefit from a dedicated illustration demonstrating ATTENTIVE l-VARIATIONAL AUTO-ENCODERS, along with the corresponding notations used in the text.

Author response:
Section 3.2 is moved to 4.2 and now includes a dedicated illustration (Figure. 3) demonstrating Attentive l-Varitiaonla Autoencodeur along with corresponding notations.

Reviewer comment (Experiments):
The authors have effectively presented the formulation for the usecase; however, in the experimental results, which I find somewhat abstract, there's a lack of a specific example illustrating how X and Y values for a time window are displayed, along with different rows of Y, etc.

Author response:
Experimental results section now includes specific examples illustrating how $\textbf{x}$ and $\textbf{y}$ values (see section 5, other results in the appendix) for a time window are displayed for each experiment, addressing concerns about abstractness we add the figure of mixed signal input and unmixed results (prediction).

Reviewer comment (Reproducibility):
In section 4.1, should be included how exactly augmentation is performed and how many, it is very abstract now.

Author response:
**Time-based Augmentation*. In Section 5.1 experiments, we have provided detailed insights into the augmentation process to enhance clarity. Additional algorithmic details are available in the appendix. The four augmentations were sequentially applied to both negative and positive: 1). **Crop and delay**: crop and delayed in time by 5 steps with a probability of 0.5. 2). **Channel Masks powers**: Randomly mask out each power channel (reactive, active, and apparent) with a probability of 0.4. 3). **Cutout or Masking**: Time cutout of 5 steps with a probability of 0.8. 4). **Add Gaussian Noise**: Introduces random Gaussian noise to window activation  $y_{m}$ and $\mathbf{x}_{m}$ with a standard deviation of 0.1. We add a comparison of performance between different choices of augmentation.

Reviewer comment (Reproducibility):
In Section 2, specifically concerning contrastive learning, the evaluation of appliance dissimilarity in "x" and "x-" is not explicitly clarified. Is labeling used for this purpose? What if the appliances are not the exact same but should exhibit similar behavior? How are such cases addressed? Additionally, the preparation of negative and positive samples is not detailed. Have you considered ensuring that there are no common or similar appliances in these two sets, and if so, how was this determined? Providing further explanation or an illustration could enhance the clarity of data preparation, which is a crucial aspect of the methodology.

Author response:
**Negative and Positive Samples.** We define a positive sample $\textbf{x}$ as a mixed power state where an appliance \(m\) is activated (\(y_m > 0\)), and `\(\textbf{x}^{+}\)` represents its augmentation. For this setup, labeled data is required. Finding a negative sample is comparatively straightforward, as any other sample $\textbf{x}$ can be chosen where $m$ is not activated $y_{m}=0$, but at least one other appliance is activated (no need for their labels). Augmentations (see our answer below) of both positive and negative samples generate sets for positives and negatives for appliance $m$. This process is repeated for $M$ appliances as preparation data.  
Given access to labels for each appliance, we meticulously consider the correlation and similarity among appliances, assessing whether they exhibit similarities, correlations, or are uncorrelated. In response to your feedback, we elaborate on this aspect in the revised version and explicitly specify the number of samples utilized for training.  
Section.3 now provides explicit clarification on the evaluation of appliance dissimilarity in $\mathbf{x}$ and $\mathbf{x}^{+}$ addressing concerns about labeling and similar behavior. Further details on the preparation of negative and positive samples have been included, along with an illustration to enhance clarity.

Reviewer comment (Experiments):
How many training examples did you use for linking? “We link the learned latent representation to ground-truth attributes using a limited number of pair labels.”

Author response:
Given access to labels for each appliance, we meticulously consider the correlation and similarity among appliances, assessing whether they exhibit similarities, correlations, or are uncorrelated. In response to your feedback, we elaborate on this aspect in the revised version and explicitly specify the number of samples utilized for training.

Reviewer comment (Writing):
After equation 2, in this test, the latent variable is represented as "z," which is defined as a matrix of dimensions (M + K) × dz, where "K" and "dz" should be introduced and defined.

Author response:
After Equation. 2, we have introduced and defined $K$ and $d_{z}$ fixed before training ($K$ unknowing appliance to find) in the revised manuscript.