# EGMStream Webapp: EGMS Data Downstream Solution
## 2 Materials and Methods
### Webapp Interface and Relative Command in the Backend  
The frontend serves as the user-facing interface of the system, designed to facilitate interactions between the user and the backend functionalities of the application. Built using HTML (Hypertext Markup Language), CSS (Cascading Style Sheets), and JavaScript languages, the interface provides an intuitive platform that is accessible via any web browser. By allowing users to make specific selections through the interface, the webapp provides flexibility, enabling users to obtain tailored results. The backend, which is the core engine of the application, manages the application logic, data processing, and user requests. Developed using Python and the Flask framework [54], the backend ensures efficient performance by supporting asynchronous requests and facilitating multiuser interactions. This robust architecture enables real-time data processing, while Java-based parallel processing optimizes computationally demanding tasks, delivering high performance and scalability for managing large geospatial datasets. The interface of the webapp consists of two main panels:  
* the left panel, where users can upload mandatory and optional input files and set parameters to customise the data conversion (Figure 2);
* the right panel, which is an interactive map displaying the uploaded AoI, if provided. The first box of the left panel is dedicated to uploading the mandatory _.txt_ file containing the EGMS hyperlinks. Through the \"_Uploadtxt_\" button, users can browse their personal computer directories to select the previously stored _.txt_ file. The file upload functionality is implemented in Python, employing the \"pandas\" package [55; 56], a robust and flexible open-source package used for reading the text file and organizing the URLs into a structured DataFrame [57]. Subsequently, the \"request\" package [58; 59] allows for managing the HTTP library, issuing HTTP GET requests [60] for downloading the assorted files and saving them in a designated directory with a name derived from their corresponding URLs. The second configuration option, \"_Crop Data on Area of Interest_\", allows users to define the optional input file specifying the AoI. If the user selects \"Yes\", an additional section appears, enabling the upload of a geospatial file in _.zip_ (containing all the files of a shapefile), _kml_, or _kmz_ (Google Earth file) formats. The handling of AoI file uploads is implemented in Python, leveraging the \"geopandas\" [61] and \"shapely\" [62] packages for efficient processing of geospatial data. If the user decides not to crop data, the process can continue with the \"_Include Time Series_\" option to determine whether time series data should be included in the final products. By default, the \"No\" option is selected. By selecting \"Yes\", an additional setting is displayed, allowing users to define the date format, namely day, month, year (choosing \"Dddmmyyyy\") or year, month, day (choosing \"Dyyyymmdd\"). In the end, the format of the converted products can be defined by choosing between the following: (i), Shapefile (_.sltp_), an ESRI geospatial format [63]; (ii) GeoPackage (_.spkg_), an open format of geospatial datasets [64]; and (iii) GeoJSON (Geospatial Javascript Object Notation--_geosion_) [65]. The GeoJSON format was introduced considering that it is a more compact format, reducing data transfer times over networks and improving system efficiency, especially in bandwidth-constrained environments. Temporal data can be easily structured in GeoJSON format, allowing more intuitive and flexible access to time series of ground displacements. Moreover, the Shapefile format has a size limitation of a maximum of 2 Gb. If the dataset exceeds this limit, it is automatically divided into smaller parts according to specific rules selected after several attempts:  
* 5,000,000 points for products without time series;
* 300,000 points for products with time series.