## 3 Results
### Revisiting Our Research Questions
#### 3.3.2 How Has the COVID-19 Pandemic Affected Application Development  
As mentioned in the introduction, the COVID-19 Pandemic pushed XPRIZE to focus on new challenges and to accelerate development. Another major effect of the pandemic was on the requirements. Another way to look at its influence is to look at the \"non-requirements\" for the Data Collaborative, which were not priorities because of the need to implement relatively quickly:  
1. **Portability between Clouds**: Extensive work has been done try to create technology that makes applications portable between Cloud Service Providers [43] and avoiding vendor lock-in [44; 45]. This goal was simply not a priority in our efforts to get the Data Collaborative up and running. We chose to use the Cloud Service Provider that we were most familiar with to get the service to production as soon as possible.
2. **Ultra-low cost**: While a reasonable cost is a key requirement (#8 above), having an extremely low cost, especially at the expense of having a usable and working solution, was not.
3. **High Performance**: While our requirement for a usable and scalable platform (requirements #1 and #2) demands enough performance to be usable, it did not demand high performance. We need the Data Collaborative in a relatively short time frame that worked and was usable. In addition, the contest time frames should allow for time for analyses to complete.  
Similar to many other IT development projects, developing the Data Collaborative had to be done remotely. There were face-to-face meetings and work sessions during late 2019, but the bulk of development was done remotely at home by the team in locations ranging from Northern California to Southern California and Arizona. We would occasionally encounter problems in working with our Cloud Service Provider, but it was impossible to know whether these were from Cloud Service Provider capacity problems or just network congestion from stay-at-home orders [46].  
\begin{table}
\begin{tabular}{l c c} \hline
**Goal** & **Goal Met?** & **Title 3** \\ \hline
1. Scalable Infrastructure & Yes & Scaled to 300 \\
2. Usable Analytics Platform & Yes & Users for one complete competition and one in progress \\
3. Isolated Secure Platform & Possibly & Secure as far as we know—no incidents so far \\
4. Scalable Analytics Compute Platform & Probably & Put in place but not yet exercised \\ (self-service capacity adds) & & Access to datasets is controlled \\
5. Control of data & Yes & Manageable by small XPRIZE team \\
6. Manageability & Yes & Started in late 2019 and first competition started in mid-2020 \\
7. Fast Implementation & Yes & No evidence of cost issues at this point \\
8. Cost-controlled & Yes & \\ \hline \end{tabular}
\end{table}
Table 1: Data Collaborative high-level requirements and whether they were met.  
3.3 Are There Any Noticeable Trends in Cloud Computing Deployment That Became Apparent during the Development?  
In addition to our observation about using multiple Cloud technologies, we have observed that even in the short time span of this project, Cloud Service Provider capabilities evolve extremely rapidly. We implemented the proxy servers because the application gateways lack certain functionality in rewriting URLs. By our second competition, the application gateway had already gained some of the functionality that was lacking, and we used it instead.