# A Query Understanding Framework for Earth Data Discovery
## 6 Results and Discussion
### Quantitative Evaluation on a Set of Synthetic Queries  
Table 1 shows part of the synthetic queries and the intermediate results of each step. The first column lists original queries. The next five columns show spatial bounding box, time range, phrase, entity, and expanded phrases in turn. Note that N/A in the table means the original query does not contain the corresponding field, and W/A indicates that the query has words or terms belonging to that field, but the tool does not detect them, which will be evaluated as \"bad\", since the information has not been successfully discovered. If the spatial bounding box and temporal range are correctly detected, their values will be converted to longitude, latitude, and time object. The concept recognition column shows the phrases that were extracted from the query. In the table, most phrases are accurately detected, such as \"ocean waves\" and \"Pacific Ocean\". Name entity recognition column displays all user-defined entities that were recognized from the query. In the name entity recognition column, the pre-trained NER model recognizes entities from most queries successfully. For example, \"quikscat\" and \"ghrsst\" were recognized as \"collection\" entity and \"sea surface temperature\" is detected as \"variable\" entity, but, in the last row, the model failed to detect \"sea surface topography\" as a \"parameter\" entity.  
Figure 8: Query understanding result of a sample query.  
Query expansion column demonstrates phrases whose similarity to phrases extracted from original query are not smaller than a predefined threshold. The higher the threshold, the less noise and fewer phrases will be added into the expansion list. In our experiment, the threshold is set to 0.9 and these phrases with a similarity larger than 0.9 to extracted phrases in the query were added to the original query for query expansion.  
A stacked bar graph is used to divide the evaluation results and compare the parts to the whole. Each bar in the chart represents the whole evaluation data, and segments in the bar represent data falling in different categories (Figure 9). Removing the queries that lack the corresponding field for each metric (Figure 10) shows that the percentages of evaluation labeled as \"good\" and \"excellent\" are larger than 80%, except for spatial range, which indicates that the proposed query understanding tool performs well. Although more than 50% of spatial parsing result is marked as \"bad\", all errors are caused by the same issue (i.e., the spatial term \"global\" in the query cannot be parsed to coordinates by the Google Geocoding API). Similarly, all of the time range terms are converted to time objects, because all of them have the same formats as \"2014-08-31t19:30:00.000z to 2014-09-20t19:30:00.000z\". The phrases are sucessfully recognized in about 90% queries, since the bigram and trigram models detect most phrases in the corpus. Customized entities are accurately recognized from around 80% queries. More than 70% query expansion results are regarded as \"good\" or \"excellent\", and the quality of query expansion depends on the frequency a query occurs in the metadata and Web usage logs. In summary, the overall percentage of \"excellent\" and \"good\" for each metric indicates that the proposed query understanding framework captures the intent of a query.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Synthetic Query** & **Spatial** & **Temporal** & **Concept** & **Name Entity** & ***Query** \\
**guikscat** 08/31/201419:30:00.000z & **\begin{tabular}{c} **N**/A \\ 09/20/201419:30:00.000z \\ \end{tabular} & \begin{tabular}{c} 08/31/2014:19:30 \\ to \\ 09/20/201419:30 \\ \end{tabular} & \begin{tabular}{c} 08/31/2014:19:30 \\ to \\ (collection) \\ \end{tabular} & \
\begin{tabular}{c} 08/31/2014:19:30 \\ (0.93) \\ \end{tabular} \\  & & & & & sea surface temperature (1.0), group \\ ghrsst global & **
\begin{tabular}{c} **W**/A \\