## 2 Background and Related Work  
### Web Scraping  
Web scraping can also be defined as a process that involves extracting and combining contents from the Internet in a systematic way, where a software agent mimics the human experience of Web browsing interaction [10]. From an operational point of view, it can be considered a manual activity, which implies copying and pasting the information. The difference here is that this task is performed in an automatic way using a virtual computer agent [11]. It might be also considered an extraction process that transforms unstructured information from the Internet into a structured format that can be easily read and used in different analyses. The web scraper accesses the web site using a request, finds specific elements using the HTML code, extracts and saves them into a structured format using a data frame. This technique of extracting unstructured data from the web sites can be applied in different scopes. For example, it can be used to compare product prices across multiple websites, to extract ads from linked pages or to cover incomplete data issues from statistics or to complete official datasets with more information [8; 12]. Regarding the human resources area, web scraping can also be used to find available jobs on different websites and to classify them using Naive Bayes algorithms. According to [PERSON], there are several advantages for web scraping. One first advantage is the low costs, as it can be free depending on the source program that is used. It can also be used anytime, depending on how frequently data need to be refreshed (hourly, daily or monthly). Another advantage might be that it can be customized according to the needs of the data analyst--compared to different datasets already existing that have predefined columns and data types [12]. Web scraping might be the easiest solution to extract datasets from the Internet, but it also comes with some challenges. One of them is that the web scraper is conditioned by the number of records available on a single page. For example, there are websites with only ten entries per page and it makes it difficult for the web scraper to go through all pages in order to extract data. We can also mention websites configured with infinite scrolling without reaching the end of the page [13]. Another limitation would be regarding the volume of the data and how long it will take a software program to extract all data if it has to extract all records each time or only new/delta entries. Data quality is another aspect that affects the web scraping process. In order to keep the accuracy of data, there should be different validation rules included in order to make sure that all the information is consistent and can be used in further analyses. Web scrapers can be implemented using several approaches, depending on the requirements received. They can be divided into two main categories--manual and automated web scraping. The manual category represents extracting data from the Internet by copying the content and pasting it into a structured format. It is the simplest form of web scraping, but it might take longer depending on the size of the dataset that needs to be retrieved. There are multiple automated web scraping techniques, such as HTTP programming, HTML parsing or DOM parsing; all of these involve navigating through the website structures and extracting the information. Different web scraping software tools can also be used in order to avoid the need to write the program and test it. Another interesting approach is using computer vision web-page analyzers that identify different blocks of content of the web page and extracts the information that is needed [11; 14]. Several libraries can be implemented in Python in order to extract data. BeautifulSoup is the easiest to use and implement, but it is not available to be used for JavaScript websites. For those websites, Selenium should be used, which is slower, as it imitates human behavior by navigating through the pages and extracting the information.