MVFL: Multivariate Vertical Federated Learning for Time-Series Forecasting

Authors: Xicun Yang, JunePyo Jung, Jialiang LU, Keun-Woo Lim, Leonardo Linguaglossa
Primary area: learning on time series and dynamical systems
Keywords: vertical federated learning, multivariate time series forecasting, resource-limited devices

Abstract:
Extending multivariate time series forecasting to resource-limited devices is a critical demand for real applications, especially with the advancements in IoT technologies. A common scenario is where the variates are distributed vertically on different devices and each device needs to do local forecasting. This paper studies a resource-efficient solution for this scenario based on vertical federated learning (VFL). Prior VFL frameworks are designed for situations where only one party holds the labels and would struggle to meet the demand of the targeted scenario, as storage resources usage would increase dramatically with the number of devices. Going beyond VFL, we design multivariate vertical federated learning (MVFL) as a novel federated learning framework, where we separate communication features and local features in an embedded feature space. This design enables MVFL to utilize storage and communication resources more efficiently by eliminating the redundant models. MVFL outperforms VFL approaches in both efficiency and accuracy. On four real-world benchmarks, compared to VFL, when the storage resources are equally utilized, MVFL yields a 12.1\% relative improvement on loss with a 43\% relative improvement on communication resources usage. Even when both MVFL and VFL employ the same main model size, MVFL achieves a 75\% reduction in storage resources compared to VFL while maintaining the loss at the same level of VFL.

Decision: reject
Reviewer scores: 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1730187090970

thread_type:
official_review

<thread_object>
creator: reviewer_mgtm

<content>
summary:
This paper proposes Multivariate Vertical Federated Learning (MVFL) to address the challenges of resource-efficient multivariate time series forecasting on resource-constrained IoT devices. By revisiting traditional Vertical Federated Learning (VFL) in the targeted scenario where each device must perform local forecasting with vertically partitioned data, this paper identified its limitations with high storage space on local devices. The proposed MVFL framework separates communication features and local features in an embedded space, offering a more efficient approach with better performances. This also prevents a rapid linear increase of model size with the increase of devices.

strengths:
1.	The structure of this paper is clear and reasonable.
2.	This paper is easy to follow.

weaknesses:
1.	The literature review of this paper is a little limited. The literature review would be enhanced to include multi-modal FL methods as well as efficient FL methods.
2.	The FL methods associated with multimodality also seem to be applicable to the case where the variables are distributed across different clients. It would be useful to have a discussion of these methods and to describe the differences and advantages of the proposed methods.
3.	It is not clear how to segment the communication and local features. Despite Section 4.3 attempts to explain the segmentation of the communication and local features, however, the specifics of this section focus on how to calculate the gradients of the local and communication features, rather than segmentation.
4.	The baselines used in the experiment is not sufficiently representative of the SOTA approach. It would be better to include more fresh and relevant work in the comparison experiments, especially in support of efficient VFL methods that deal with multiple variables.

questions:
What are the differences and connections between “multivariate” VFL and “multimodal” VFL?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730645916364

thread_type:
official_review

<thread_object>
creator: reviewer_n5rj

<content>
summary:
The paper introduces MVFL, a novel federated learning framework for multivariate time-series forecasting on resource-limited devices. It addresses the inefficiency of traditional VFL by separating communication and local features in an embedded space, reducing storage and communication costs. The methodology involves a modified VFL approach with a focus on efficient resource use, demonstrated through experiments on real-world benchmarks.

strengths:
1. Originality:
The paper presents a novel approach to federated learning that is tailored for time-series forecasting, which is a creative extension of existing VFL frameworks.
2. Quality:
The experiments are well-designed and demonstrate the effectiveness of MVFL in terms of efficiency and accuracy.
3. Clarity:
The paper is clearly written, with a logical flow that makes the complex concepts accessible.
4. Significance:
The work addresses a significant problem in IoT and federated learning, with implications for practical applications. It also provide insight for possible future more efficient multivariate time series forecasting solutions.

weaknesses:
While this work presents a novel method with significant contributions to the field of multivariate time-series forecasting on resource-limited devices, there are areas where the work could be improved:

Though the novelty of the proposed model is promising, its applicability on real-world datasets could be further validated with a broader range of diverse datasets. This work aims to address the challenges of multivariate time series forecasting; however, the datasets used do not adequately demonstrate its effectiveness in this regard. The largest dataset they used, Weather, includes only 21 variables, whereas real-world time series data often involves far more complex variables, as seen in datasets such as Traffic (862 variables), ECL (321 variables), and PEMS07 (883 variables).

Additionally, have the authors considered evaluation metrics other than MSE and MAE? The R-squared (the coefficient of determination) is a quite commonly used evaluation metric in time-series forecasting. While MSE is valuable for understanding prediction errors, R-squared provides a broader view of how well the model captures the underlying relationships in the data.

questions:
1. Are there any real-world application scenarios where MVFL has been tested or is planned to be tested？The authors could discuss the potential challenges and benefits of deploying MVFL in practical IoT environments.

2. How sensitive is the performance of MVFL to the choice of forecasting model? Would the authors share any insights or best practices ?

3. Besides, the authors could investigate the use of interpretable forecasting paradigms like N-BEATS, or decomposition strategies such as Seasonal and Trend decomposition used in Autoformer and Leddam, to enhance the transparency of their model and provide insights into the patterns learned from data.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730708205072

thread_type:
official_review

<thread_object>
creator: reviewer_yhdr

<content>
summary:
The authors propose an approach MVFL to forecast the time series in resource-constraint IoT scenarios. They separated communication features and local features in the embedded feature space and used storage and communication resources more efficiently by eliminating redundant models. Experiments on efficiency and accuracy are conducted to evaluate the performance of the proposed mechanism.

strengths:
1. The considered scenario is novel. Considering limited resources in federated learning for time-series analysis is significant.
2. Experiments show that the performance of the proposed method is significantly better than existing methods in different scenarios.
3. The paper is generally well presented.

weaknesses:
1. Traditional VFL has different label spaces between clients. The time-series forecasting proposed in this article is not a classification task, so is it a reasonable scenario for each client to hold the features of one dimension in a multi-dimensional time-series? Will there be such an application in reality? In addition to the weather dataset, will other common time-series forecasting datasets such as Electricity, Traffic, and ILI also exhibit such a scenario? It seems to be a synthetic scenario. 

2. How can privacy issues of communication features be preserved?

3. The adopted model is unclear. What about the trendy transformer-based model or other time series forecasting models, like PatchTST, TimesNet, FEDformer, Informer or others?

4. The comparisons in experiments are insufficient. The proposed methods are only compared with VFL. It can be compared with other FL for time-series forecasting like MetePFL[1] or VFL work (changing the downstream task). Only one baseline cannot validate the effectiveness of the proposed method.

[1] Chen, Shengchao, et al. "Prompt federated learning for weather forecasting: toward foundation models on meteorological data." in IJCAI 2023.

questions:
Refer to weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732533804154

thread_type:
official_comment

<thread_object>
creator: area_chair_y5rw

<content>
title:
Acknowledge the author responses

comment:
Dear Reviewers,

Thank you very much for your effort. As the discussion period is coming to an end, please acknowledge the author responses and adjust the rating if necessary.

Sincerely,
AC
</content>
</thread_object>

</thread>
<thread>
creation date:
1732797458229

thread_type:
official_comment

<thread_object>
creator: area_chair_y5rw

<content>
title:
Discussion needed

comment:
Dear Reviewers,

As you are aware, the discussion period has been extended until December 2. Therefore, I strongly urge you to participate in the discussion as soon as possible if you have not yet had the opportunity to read the authors' response and engage in a discussion with them. Thank you very much.

Sincerely,
Area Chair
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper studies a resource-efficient multivariate time-series forecasting based on vertical federated learning (VFL).  Although the paper presents an interesting, important application scenario, the reviewers agreed that the merit of the proposed method is not fully proven through convincing evaluation.  Multiple reviewers pointed out that the novelty is not convincingly presented.  Thus, I would like to recommend a reject.