# Big Data Integration in Remote Sensing across a Distributed Metadata-Based Spatial Infrastructure
## 4 Spatial Organization and Management of Remote Sensing Data
### Full-Text Index of Multi-Sourced Remote Sensing Metadata  
After spatial organization of multi-source remote sensing data, the full-text index of metadata should be constructed to enable quick retrieval. It should be added that, as the query index of remote sensing data involves many terms, the column-oriented key-value data store, like HBase, cannot effectively handle multi-condition joint retrieval. Hence, in this paper, the multi-sourced remote sensing metadata retrieval used the full-text index, and its construction was mainly implemented by Lucene and SolrCloud. In essence, Lucene is a high-performance, full-featured text search engine library written entirely in Java, and the ready-to-use search platform provided by SolrCloud is also based on Lucene. Lucene supports the full-text index construction of static metadata fields and dynamic domain fields. However, Lucene is not a complete full-text search engine; it should be combined with Solr or SolrCloud to provide a complete search service [39]. SolrCloud supports the following features: (1) central configuration for the entire cluster; (2) automatic load balancing and failover for queries; and (3) near real-time search [40,41]. SolrCloud uses ZooKeeper to manage these locations, depending on configuration files and schemas, without a master node to allocate nodes, shards, and replicas. Each node runs one or more collections, and a collection holds one or more shards. Each shard can be replicated among the nodes. Queries and updates can be sent to any server. Solr uses the information in the ZooKeeper database to figure out which servers need to handle the request. Once the SolrCloud cluster starts, one of the nodes is selected as a leader, which is responsible for all shards [42]. In addition, there is a master controller in the cluster, called the overseer node, which is responsible for maintaining cluster state information and thereby provides for failover to the Solr cluster (Figure 6). Figure 5: The virtual mapping between the logical partition indexing and the scene parameters of remote sensing data. In this study, the distributed index based on SolrCloud/Lucene was applied on the static and dynamic metadata fields. As the name suggests, the distributed index will be used when our index collections are so large that we cannot construct an index efficiently on a single machine. The static remote sensing metadata fields are defined the by OODT file manager, and include ID, CAS.ProductId, CAS.ProductTypeName, CAS.ProductReceivedTime, and CAS.ProductTransferStatus, CAS.ProductName, CAS.ProductStructure, and so on. The dynamic domain fields mainly include the satellite type, sensor type, scanning time, and GeoSOT codes. In addition, in order for the comparative experiments, the original latitude and longitude of each image is also included in the dynamic domain fields. The static and dynamic fields' index are as shown in Table 2. It is noted that in Table 2, the asterisk (*) denotes all of the dynamic domain fields of the remote sensing metadata. The dynamic domain fields are implemented in Lucene by adding the 'text' and 'text_rev' fields to the full-text index field. These two fields are copies of all the dynamic domain fields. Their purpose is to implement multi-granularity segmentation for the dynamic domain fields. Therefore, the following configuration should be added in schema.xml of Lucene. <copyField source=\"\"dest=\"text\" />  
<copyField source=\"\"dest=\"text_rev\" />