## 4 Mobility and Environmental Data Integration: System Overview
### System Architecture
#### 4.2.2 Spatial Join Processor  
After integrated data pours downstream toward the storage, the spatial join processor starts its operation to preprocess data and put it in a unified view format. This component is divided into two logical components: geohash encoder and join processor.  
The geohash encoder receives pollution, meteorological and mobility data and transforms locational coordinates information to a single-dimension string representation known as geohash. Geohash encoding accepts a pair of geographic coordinates (i.e., longitude and latitudes) and transforms them to a geometric point. Then, the component takes as input an integer that indicates the precision of geohash encoding and creates a list of Z-Order curves for referencing each geometric point. Subsequently the indexes of the Z-Order curves are hashed to base 32 and exploded from lists to single elements in each record. At the end of this process, each tuple of the datasets is enriched with a geographical location expressed as a geometric point and its relative geohash calculated with the specified precision. The geohash uniquely represents the rectangle of the grid that contains the location referenced by the point. This step is introduced for transforming the handling of geographical locations from a two-dimensional space to a one-dimensional counterpart. In this way, the spatial join comes down to an equijoin with equality conditions on string geohash representation.  
The same geohash encoding is applied to pollution, meteorological and mobility data. On map data, GeoJSON/shapefiles is imported. After reading the data, the geohash encoding is applied to each polygon and a list of geohash values covering each polygon is generated as shown in Figure 6, in accordance with Definition 2 from Section 4.1.  
Having the two datasets enriched with geohash codes, the system continues operation as follows. The filter step of the stock filter-refine is applied to get the pairs in three classes, true and false hits, in addition to remaining candidates. True hits are maintained, false hits are discarded, while the remaining tuples are forwarded to a refine stage that is a retrofitted, repurposed, and adapted version of the refine stage from the plain version. In the new version, each pair from the remaining candidates is checked with the exact geometry (ray casting for point and polygons in this case). This will result in two possibilities, points from mobility that fall in specific polygons and the counterparts form pollution and meteorological dataset. Then equijoin is performed to filter p  
Figure 6: Geohash encoding applied to the map of neighborhoods in Bologna, Italy.  
geohash values and geometrically fall within the same polygon. Those are the points that will be maintained, the others will be discarded. Figure 7 shows a heuristic overview of the adapted filter-refine method, while Figure 8 shows a toy example of such join.  
In summary, our join processor is an adapted, retrofitted, and repurposed version of the stock version of filter-and-refine approach. It first applies the plain filter-refine to each data with the polygon data (pollution-polygon, mobility-polygon) to enrich both data with geohashes and neighborhoods. Thereafter, it employs a simple equijoin on the two variables \"geohash\" and \"neighborhoods\" to simply join the two datasets into a unified pollution-mobility view that can be easily queried for combined sophisticated analytics. Our join processor works as per Definition 5.  
Figure 8: A toy example of joining pollution (or meteorological) and mobility data with the support of EMD system.  
Figure 7: Adapted and retrofitted version of filter-refine for joining heterogeneous mobility, pollution and meteorological data at scale with QoS guarantees.