## 5 Evaluation
### Test Methods  
The setting of the prototype variables \"key words\", \"number of results\" and \"domain\" were decided upon by the test person. In total, five different search queries were performed.  
The key words for each query were chosen based on the professional experience of the test person. In order to make an evaluation as comprehensive as possible, different fields of technology were explored. The following key words were used per query:  
* New multifunctional device for SME;
* Recommend ERP system for small business;
* Trends ERP system for small business;
* Mixed reality for workshops;
* Business process management tools trends.  
The top-level domain was set to \".com\". For the first query, the number of results were limited to the default value of five. Especially for the quality test of the text summarization part, it was important to see a broad number of results and therefore the other four queries were extended to a value of ten for the number of results. The number of checked websites was even further increased in two cases as some webpages do not allow web scrapping and the protype has to skip those websites. In \"trends ERP system for small business\" the number was increased to 11 results and in \"recommend ERP system for small business\" the number was increased to twenty results. This allowed the prototype to output a total of ten results for both queries.  
The human performer executed the following steps for comparison:  
1. Go to search engine www.google.com;
2. Enter search terms (key words);
3. Access the top result;
4. Extract URL to result report;
5. Find and extract relevant information to the result report;
6. Access the next result;
7. Repeat 4-6 until the desired number of results is achieved.  
Once the queries from the human and from the prototype were finished, both results were added to the evaluation sheets.  
In the evaluation sheet A, the functionality of the prototype and the quality of the URL were rated. The following questions were asked:  
* Does the prototype extract the URL it is supposed to?
* Does it adhere to the given parameters?
* In the evaluation sheet B, the quality of the text results was rated. The following questions were asked:
* Is the summarized text understandable for a human reader?
* Does the text summarization show the content of the website?
* Is the content informative? \"Informative\" means that the software provides an answer to the keyword search, and it does not mislead.  
Duplicated results were eliminated in evaluation B.  
Evaluation C aimed at the time efficiency. A new query was set up for this evaluation because the reading time of the human performer should not be affected by former tests. For this test, the variables were set \"domain = com\", \"number of results = 10\" and \"keywords = document management system for manufacturing industry\". The human performer executes steps 1-5. The following question should be answered with this test:* How long does each method need from start to the end?  
For both methods, the time was taken from the beginning of each process, i.e., for the human when starting the search engine and for the prototype when starting the program. The time was stopped when the defined number of results was scrapped and summarized. The prototype put a result page out and the human performer added their results in the result report. The speed test shows only the efficiency in terms of time, the search results and the quality of the results are not measured in this part of the evaluation.