C OVID-19, an infectious disease caused by the most recently discovered coronavirus was identified in 2019 [1] , December 31th 1 . The virus has spread worldwide in less than three months, but infected more than 5.7 million people around the world and caused over 844,000 deaths 2 . This novel coronavirus outbreak leads to severe illness and deaths in both younger and middle-aged adults, and it also critically affect people's daily lives. For example, almost 500 million schoolchildren have been cut off from learning recently for avoiding direct exposure to this disease; thousands of people canceled their trips to reduce the risks of potential infection, etc. So far, there is still no specific antiviral treatment for COVID-19. Fortunately, we observe that various active response plans have a clear positive effect upon preventing transmission from one person to another. One of the successful examples is in South Korea 3 . Vt taking early, aggressive government response including establishing a large number of detection sites, isolating the infected patients in a very short time, and tracing contacts for searching the potential infected people with thoroughness, South Korea controlled the spreading efficiently. Thus, we observed that South Korea has quickly flattened the epidemic curve after its first peak in February 2020, and the daily cases declined to nearly zero after two months. The similar observation can also be found in many other countries, such as China, Germany and the United Kingdom. That means, if we response appropriately, the infection spreading can be effectively controlled. Consequently, it is necessary to establish a paradigm to collect the historical data, and distinguish the effectiveness of various response plans. We also expect that such a paradigm could be used for other urgent city crisis. To obtain the above goal, city Digital Twin (city DT) has demonstrated a remarkable potential to offer the function of data collection, in-depth analysis and most important, it operationalizes real-world insights to asset performance and refine future actions [2] , [3] . In particular, a city digital twin is a virtual replica of the city that can provide digital view of city facilities, human activities and behavior, and other types of information about urban area. By keep interacting with the target environment, a city digital twin not only could automatically learn the patterns and generate useful insights from the historical data, but also continuously improve itself by verifying the automatic operations within the real world. Eventually, a city DT enables interdisciplinary convergence in multiple aspects of city informatics, and offers increased visibility and machine intelligence for both human assisted decisions and automatic operations to facilitate city government and planners' decision-making. Several successful attempts, including Columbus Consolidated Government (Columbus), Digital Twin City of Atlanta, and Virtual Singapore, validate city digital twin's value for capture the complex spatial and temporal implications among multiple data source to optimize urban sustainability on traffic management, architectural planning and energy saving solutions. Meanwhile, [4] proposed a novel framework of disaster city digital twin with deep learning techniques, which presents the visions for a city DT based disaster management and further explains the advantages to response the disaster and crisis. Thus, we expect that the city DT could be a promising solution to tackle current health crisis by simulating various response plan in this virtual system, and finally determine the proper actions to control the infection trend.
Note that, when we aim to construct a city digital twin to model COVID-19 pandemic and generate the prediction of a specified response plan, we need to resolve the data sparsity challenge with considerations of privacy by following reasons. First, a city digital twin for health crisis like COVID-19 may contain more sensitive information than other application scenarios, which imply the pressing need for privacy protection on the raw data. Second, as each city constructing its own digital twin system, which means it could form an isolated data island. Under such conditions, each city can only depend on a trail-error manner to control the pandemic instead of learning experience from others, which may result in delayed response or inappropriate operations. To resolve such problems, a collaborative city digital twin paradigm can be beneficial to alleviate the data sparsity issue. Meanwhile, it is also helpful on pursuing an accurate prediction for choosing a better response plan. To this definitive end, we need to answer three primary questions: (1) How to find a collaborative plan that can enhance the prediction results while not disturbing the interactions between city digital twin systems and the physical environment? (2) How to ensure the collaborative solution can automatically generate different insights to adapt to different city digital twin system? (3) How to provide the collaborative city digital twin paradigm with privacy protection guarantee? A straightforward solution is the distributed machine learning method with privacy protection guarantee, so that various city digital twin can upload their raw data to the central server to achieve a learning result based on data fusion results. However, the initiative data pre-processing stage of this plan could bring extra time expenses, and the centralized training manner is likely to have higher risks on privacy leakage and security issues.
To address the above challenges, in this paper, we propose a federated learning (FL) [5] based solution to bridge these separated city digital twin for COVID-19 pandemic control, which is depicted in Fig. 1 . In particular, each city digital twin is a client in our FL paradigm. Each client uses a Sequence2sequence (Seq2seq) model to generate the local forecasting model by historical conditions of COVID-19 in targeted area. Multiple city DTs can cooperatively train a shared model on FL platform, only with local updates without exchanging the raw data. That is, each city DT can be improved by both local data and external insights from other city DTs to make more efficient response operations. Our contributions can be summarized as follows.
• To resolve the data sparsity challenge for COVID-19 pandemic management, we are among the first to propose a novel collaborative training framework for multiple city digital twin.
• Considering the various privacy requirement among different city DT, we utilize the Federated Learning techniques as our primary solution, which offers a parametersharing only concept for not disturbing each city DT's privacy rules.
• Extensive simulations with a real dataset reveal that the our proposed framework significantly improves the performance when compared with the single DT system. The rest of the paper is organized as follows. Section II introduces related works. The basic definitions and problem statement is presented in Section III. Section IV explains the detailed structure of the proposed framwork. The experiments and results are analyzed in Section V. Finally, conclusion and future work are presented in Section VI.
Historical insights for understanding the character of different epidemics and the prediction of spreading trends have been crucially crucial for epidemic control and prevention. Several past studies applied artificial intelligence based models to these highly specialized domains and observed impressive multi-functional properties, such as the deep neural network based short-term and high-resolution epidemic forecasting for influenza-like illness [6] , the semi-supervised deep learning framework that integrates computational epidemiology and social media mining techniques for epidemic simulation called SimNest [7] and EpiRP [8] that using representational learning methods to capture the dynamic characteristics of epidemic spreading on social networks for epidemics-oriented clustering and classification. To appreciate why deep learning techniques are required, in this section, we briefly review several recent relevant works and their contributions in artificial intelligencebased urban epidemic control.
Deep Learning based Epidemic Control: For the past decade, deep learning technique has been used in a variety of smart health areas, including disease prevention and monitoring, intelligent diagnosis and treatment systems, and health decision-making. Recent breakthroughs in various disease modeling, forecasting, and real-time disease surveillance have convinced us that those applications mitigate the effects of disease outbreaks. Given various application scenarios and objectives, deep learning based models can be different: A typical solution for localized flu nowcasting and flu activity inferring is ARGONet [9] , which is a network-based approach leveraging spatio-temporal correlations across different states to improve the prediction accuracy. Based on the information from influenza-related Google search frequencies, electronic health records, and historical influenza trends, ARGONet uses a spatial network to capture the spatio-temporal correlations across different states and produces more precise retrospective estimates. Instead of leveraging multiple data source like ARGONet, [10] proposes a multi-task learning based model that only using user-generated content (Web search data). They investigate both linear and nonlinear models' capability and find they can improve the disease rate estimates significantly in the case study of influenza-like illness. These successful attempts, however, are based on large scale data sources or plenty historical information of the disease with similar spreading patterns. That means, high-dimensionality, irregularity forms, noise, or sparsity problems are possible to affect these learning-based models' performance. Such problems would be more serious when the smart health system needs to deal with unexpected infectious disease outbreaks like COVID-19, due to limited historical data and information.
City Digital Twin: For most of deep learning based applications, the performance depends on the interaction with the physical world to generate data with high quality. To fill the data gap, digital twin is proposed as a virtual representation of a device or a specific application scenario, which can interact with the target environment to collect data continuously for real-time decision making. Such a concept leads to the advancements in both urban computing paradigm and IoT applications direct extending to smart city area recently, which termed as 'city digital twin'. It is a digital model of the physical urban environment, which collects information by hundreds of IoT devices or public databases. It provides remote decision making and real-time insights about the physical asset's performance by utilizing machine learning and artificial intelligence techniques. It also serves as an essential role in addressing public health, urban planning, and environmental issues. The data streaming in the city digital twin model enables proactive remote monitoring to identify patterns and risks that foreshadow specific impending issues.
Several successful research attempts including disaster city digital twin [11] , [4] , energy management [12] , and cityscale Light Detection and Ranging (LiDAR) point clouds [13] . Furthermore, Singapore [14] and Germany [15] has launched the city-scale digital twin to monitor and improve utilities, which enhance the transparency, sustainability and availability of DT.
Federated Learning: Pandemic control and response are the significant challenges in the field of urban computing. However, recent research on digital twin is designed for the city resource management for a single urban area, and such application scenario often has enough historical data and multiple resources. Unfortunately, the data resource could be the most challenging requirement for DT aiming at pandemic control, especially the unexpected contagious disease like COVID-19 due to lacking enough experience and information.
To maintain the advantages of DT and tolerate the data sparsity challenge, federated learning that allows multiple stackholders sharing data and training global model has become a preferred scheme. In the typical FL scheme settings, each data owner (FL client) engages in a collaborative training process without transferring the raw data to the others. Through FL, the central server manages each client's local training updates and aggregates their contributions to enhance the global model's performance. Several concrete scenarios include Google's Gboard, health AI and smart banking showing the advantages of federated learning in handling collaborative training issues and data difficulties among diverse data owners. Therefore, we are motivated to utilize federated learning techniques to resolve the data sparsity challenges and design a collaborative city digital twin for COVID-19 pandemic control.
In this section, we propose our designed system model of city digital twin for Covid-19 Pandemic Control. We first introduce the basic definitions, and then give the problem statement and explain the implementation difficulties.
A city digital twin has three primary components: the physical environment of the city, a virtual replica describing the city's architecture, functions, behaviors, and active communications between the two for obtaining real-time spatiotemporal data from various infrastructure and human systems [16] . In our work, we follow this typical architecture, and propose the city digital twin for COVID-19 pandemic control using the following metrics:
COVID-19 infected number: The COVID-19 infected number is the direct evidence to represent the infection status of a specific geographic area. In our proposed DT model, the infected number in the target area is represented by: area, time, number.
Response plan: For COVID-19 pandemic control, various organizations and governments develop several local level or even a country-level operational plan to prepare for and respond to COVID-19. In our DT model, we use R i = (l i , t st , t end ) to represent a response plan, where l i is the location, with t st , t end denotes the starting time and end time of R i . The response plan includes 14 day quarantine, domestic travel limitations, gatherings limits and stay at home, nonessential business closures, reopening plans, mask policy, etc. The effectiveness of different response plan can be varied, since it is may be affected by several external factor, such as a crowd gathers, adverse weather conditions or using vaccines.
Temporal effects: For our proposed DT system, the primary goal is to analyze the specific response plan, determine whether the historical plan will impact the future infection trend, to estimate the possible plan effects. In the process of building such a virtual replica for the COVID-19 pandemic control, we observe the temporal effects is a critical factor when we aim to provide useful guidance on response plan selection. These temporal effects can be considered two types in our proposed DT system: temporal effects of historical infection numbers and selected response plans.
Temporal effects of historical infection numbers: The historical infection numbers are direct evidence to describe the correlation between past conditions and current infection status. In Fig 2, we take historical daily case information of three states (NV: Nevada, UT: Utah, WI: Wisconsin) as examples to illustrate the temporal effects of multiple factors. From the data of early March from three states, we observe an immediate impact of historical infection numbers since it leads to a continuously increasing number of infections until April 2nd, 2020. Such correlations play quite an essential role in explaining and predicting future infection trends.
Temporal effects of Response Plans: The choice of a specified response operation can also significantly impacts the number of infection. This impact could be immediate or delayed, which may depend on the strictness of that operation or people's acceptance. For example, in Fig. 2 , we observe that after taking a specified response plan, such as domestic travel limitations or gathering limits, the infection trend is decreased in the next several days. Such impact could be complicated, since the 14-day time window, the time that a response plan starts to take effect, and paroxysmal public crisis may also 
With the definite goal of making the COVID-19 pandemic under control, different local agencies in each city/region may choose their own strategy to meet their local requirements. Such divergence is mainly caused by the fact that different cities could have other conditions or expectations so that the local agency tends to provide a different response plan considering several local intrinsic properties. For instance, area A, which is a thinly populated district with very low infection rates, would prefer to choose less radical response plan; while the another area B, where has more serious infection conditions, is very likely to choose a more strict response plan like restricting activities and closing facilities. Obviously, such unsynchronized operations present obstacles to accumulating knowledge about providing accurate response guidance, since each city forms an isolated data island. Furthermore, COVID-19, as an infectious disease with unknown cause, has exaggerated the situation due to lacking enough historical information and experience for dealing with the new virus. That means, each city can only obtain knowledge by trialand-error operation schemes for seeking an effective response plan, and the increasing time cost may lead to a delayed policy with poor performance. To resolve the above difficulties, we formalize this problem as a FL training problem using the FL approach to resolve the data sparsity challenge in city digital twin construction.
Thus, we propose to learning a objective function as follows: Given multiple city DTs D = D 1 , D 2 , · · · , D i that expecting collaborations and bounded with a local data sensing method to generate individualize data source s i,1 , s i,2 , · · · , s i,mi , our federated training problem is aim to optimize following function:
where p i = mi m and the local objective F i (·) is defined by
In this section, we present our proposed method utilizing FL as the collaborative platform for predicting a response plan's effectiveness, with considerations of temporal patterns and external factor impacts. Specifically, we describe the baseline model using Seq2seq structure, and give the details of the proposed federated training process to help understand how every single city DT can be automatically improved by such collaboration.
A typical city digital twin actively collects real-time data from infrastructure and human systems to obtain automatic decision making or possible future behavior predictions. Hence, it is beneficial for tracking the progress of the infectious disease in real-time and accumulating information and knowledge at city scale. However, COVID-19 is a novel coronavirus with incredible spreading speed, which means the city DT faces the challenge of making effective operations with very limited response time. Under such circumstances, single city DT likely lacks experience to determine a well strategic preparedness and response plan quickly since each city DT has limited number of actions and observations. Thus, we introduce FL as the primary solution to implement a collaborative training process, enabling multiple city DTs to train a global model for response operation prediction collaboratively.
Federated Learning is a novel distributed learning framework with a privacy protection guarantee. Unlike previous research dealing with training data in a centralized manner, FL uses a parameter only concept to avoid disturbing each collaborator's privacy rules. Thus, multiple collaborators can learn a common model as partners while preserving personal data on their own devices. In particular, for a new FL training task, (1) the FL central server would train a global model for initialization, then distributed this model to the existing collaborators (clients); (2) after receiving the global model, each collaborator use the local dataset to update the local parameter and generates the local updates; (3) based on specified synchronization settings, all these updates are back to the FL central server for aggregation, and the global model is improved; (4) such distributed updates iteration are From the above facts, we observe clear advantages of utilizing FL for establishing the collaborative city DTs framework: (i) by the separation of local model training and global model updates, FL offers a strong capability to deal with the isolated data island problem between multiple DTs; (ii) with enhanced privacy settings, each city DT can obtain the collaboration achievements without violating its privacy rules.
In our proposed collaborative framework for multiple city DTs (Fig. 3) , each single city DT system can reinforce itself by two ways: (1) self-renewal mode by local updates: this mode is similar as the working updates of traditional digital twin system, which collecting real-time COVID-19 conditions (historical infection numbers, external factors like crowd gathering, population age and vaccination) actively through on-device sensing, or using data source from various institutions like public health agency or hospital. (2) jointrenewal mode by global updates: our proposed FL framework offers a platform for all City DTs with willingness for sharing its experience and insights. By such collaboration, the single city DT can gather information from others to update the local model, which we regard as a joint-renewal mode in this paper. Specifically, our works aim to simulate a response operation's performance for guiding choosing an appropriate response plan of COVID-19.
Under such a practical context, the joint-renewal mode can effectively complete the local models by utilizing other collaborator's updates. For example, suppose a City DT A decides to take a new response operation R i to deal with a crowd-gathering caused infection outbreak; at that time, City DT B, which have used the same response operation R i before, can broadcast its experience through the FL platform by uploading the local updates on time. Then, after obtaining 
For the local City DT system, we utilize the GRU based Seq2seq learning model with Gated Recurrent Unit (GRU) for local training, which is illustrated by Fig. 4 . The Seq2seq learning pipeline is one of the typical Encoder-Decoder structure to solve the general sequence to sequence learning problem with neural networks, which has well performance when input and the output sequences have unfixed lengths with complicated and non-monotonic relationship [17] . The GRU layer is a variant of Recurrent Neural Networks (RNN) for better time-series data processing, and each GRU cell only uses two data gates to control the data from processor: the reset gate r determine the information needs forget, and update gate z decides how the information needs to be updated. In particular, our proposed local City DT system using threelayer GRU for the encoder part, which maps the history of sequences (the infection number of the target area and response operation) to latent representations. After the encoder part processing the history data of 14 days, these hidden states pass to the decoder part. The decoder using GRU and three full connected layer, which accept the hidden state from encoder as initial state and the expected response operation as input, while the output is the prediction of the infection number at next time unit. Eventually, the local city DT would obtain the infection trend's prediction result in the next 7 days.
The federated training process aims to provide a collaborative training protocol without violating the privacy rules of each City DT, and the whole training diagram is depicted in Fig. 5 . This training process includes following steps: (1) In the beginning, the FL central server trains a global model by pubic data source or voluntary data set from
Step 1: Initialization
Step 2: Local model training and upload
Step3: Aggregate and update global model
Step 4: Deployment dataset and experiment settings of the framework. Then, multiple aspects of experiment results, comparisons and analysis are provided, especially the comparisons between non-federated methods and our proposed method in terms of prediction accuracy.
Dataset Description: We used the covid tracking project dataset 4 and the Coronavirus (COVID-19) State Actions Dataset 5 to conduct experiments. The COVID tracking project dataset contains each state's daily epidemic data from the first case in the United States from January, 2020 to July, 2020, including information on the number of nucleic acid tests, the number of confirmed cases, and the number of severe cases. The Coronavirus (COVID-19) State Actions Dataset mainly contains specific information about the policies adopted by various states in the United States during the epidemic, including the policy's name, the start time, and the end time. In this paper, we also need to consider other external factors, such as the change in the number of infections in each state in a time zone, the implementation of various response operations, etc. Therefore, we combine the two data sets for our model training. In our combined dataset,
• State name indicates the state where the current data is located.
• Date represents the current data time.
• Data − (t − 13, · · · , t + 7) contains a total of 14 days of epidemic data starting at time t (13 days forward and 7 days backward), and each day's data is determined by the number of people diagnosed and the current response operation. A one-hot vector represents every response operation, if it is selected, it is represented as 1; and if not selected, it would be represented as 0. To eliminate the epidemic data's volatility and highlight the inherent time trend and periodicity, we use the moving average method to calculate the moving average of the number of infections for each day, and the moving window is set to 7.
Experiments settings and Parameters We conduct all the experiments on the PyTorch platform, and use PySyft [18] framework for the implementation of our FL protocol. For the baseline model for local DT training, the encoder is with three-layer GRU and the decoder is with three fully connected layer and a three-layer GRU. We set the learning rates of the neural networks as 0.005, mini-batch size as 60, and evaluate the accuracy of prediction results with Mean Square Error (MSE), which is defined as follows:
In Fig. 6 , we show the comparison results of our federated solution and non-federated solution. It shows the difference of the loss value of both of the two models in 22 states, each using the individual training method and federated learning method. For our federated learning model, FedAvg is adopted as the aggregation algorithm, due to the considerations of saving communication resource. The selected response operation includes domestic travel limitations, gatherings limits and stay at home, nonessential business closures, reopening plans and task forces, statewide mask policy. As shown in Fig. 6 , in most states, the loss of epidemic prediction of FL training is much less than that of single DT models. This indicates that through FL training, each city DT can exchange the epidemic information of their own city to improve their DT model, so that the prediction performance for COVID-19 trend under selected response operations maintains at a relatively high value. Fig. 7 shows the loss rate of the city DT model from six states (ID: Idaho, MT: Montana, TX: Texas, CO:Colorado, ND: North Dakota, IL: Illinois) with the change of the number of FL clients (collaborators). The steel-blue line of each city DT in Fig. 7 represents the loss value of the prediction result based on single DT model that only rely on its own dataset, so that it will not change with the number of participating FL members. The dark-orange curve shows the loss value of the city DT model that training in the FL environment. When the number of people participating in the FL platform is increasing, it can be observed the performance has been gradually enhanced, since the information obtained by DT in each city has gradually accumulated. Under such conditions, the prediction results have become more accurate. Thus, the experimental results show that increasing the number of FL clients can effectively improve the training effect and reduce the prediction error of each single city DT.
VI. CONCLUSION In this paper, we propose a novel collaborative city digital twin framework based on federated learning techniques for COVID-19 response plan management, with Seq2seq structure to better capture temporal contexts in historical infection data. The proposed framework shows that the collaborative learning of multiple city digital twin is beneficial for alleviating the data sparsity challenge, especially when the novel coronavirus management needs more data sources and highly strict privacy protection requirements. Besides, the experiments validate our approach achieves better performance on the real dataset. In the future, we aim to involve more data sources (e.g., movement of people, seasonal changes, temperature, and humidity) to improve the performance of the proposed framework. 