## 3 Methods and Results
### Data Collection
#### 3.2.1 Web Scraping Process  
The web scraping process that extracts data as reviews from different websites can be observed below in Figure 1. It was developed using the BeautifulSoup library to get the information and parse the HTML code and request library to send HTTP calls for the website to retrieve specific data. The first two steps describe the website and the data that are needed for the web scraping process. The extracted source entities (such as names, descriptions, and links) are stored in the web page structure, so they will be retrieved from the code source using HTML DOM or XPath methods. Those identifiers will be stored in a metadata table to be parsed as parameters. This will allow for using the same code source for different websites even if they have different identifiers for a specific element--for example, the title of the review is extracted from the h2 tags that have the itemprop named \"author\" for the Indeed.com website. On the other hand, the rating is available for all the reviews found on the Glassdoor website using the span selector with the class called \"authorInfo\". The structure of the table can be analyzed below (see Table 1). This table allows one to use parameters in order to define the company and the website for which the user wants to extract data--in this, the source code will not be modified. In Figure 2, there can be seen the structure of a review that corresponds to the function used to extract the elements--the review, title, and feedback. The author is a string that contains multiple elements, such as job function, job status (if it is a current employee or not), location, and the review data. The fourth step is to create the source code that extracts data from the website. It connects to the URL added as a parameter to automate the process to be used for multiple \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Website** & **URL** & **Element** & **Selector** & **Type** & **Value** \\ \hline Indeed.com & [[https://www.indeed.com/cmp/Amazon.com/reviews](https://www.indeed.com/cmp/Amazon.com/reviews)]([https://www.indeed.com/cmp/Amazon.com/reviews](https://www.indeed.com/cmp/Amazon.com/reviews)) & author & h2 & itemprop & author \\ \hline Glassdoor & [[https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm](https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm)]([https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm](https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm)) & author & span & class & authorInfo \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Metadata table. Figure 1: Process of web scraping (source: authors own study). Figure 2: Structure of a review: (**a**) Indeed.com; (**b**) Glassdoor. companies and creates a request to access data. If the connection is successful, a list is created, and it navigates through all pages to extract the reviews. The last step of the web scraping process is the transformation of data into a structured format that can be used for further analysis. The title column was split into multiple columns to retrieve the job name, location, employee status, and the data; all columns were converted into appropriate formats to be further used in the analysis. The approach of using metadata and parameters improves the scalability of the process as there can be used the same code script to extract data from multiple websites or to use the same extract but retrieve data for multiple companies. However, the disadvantage of this approach is that the process is slower when we are using parameters. In Table 2, there can be observed the timings obtained while running the web scraping script using parameters instead and the values using Databricks. This is a resource provided by Microsoft Azure and it is used in order to process large workloads of data by running queries in different languages, such as Python, R, Scala or SQL. For the compute power, it uses a cluster with 14 GB memory and 4 cores, so there will be no need for additional infrastructure to process data.