# RESTful API Testing Methodologies: Rationale, Challenges, and Solution Directions
## 4 Results  
This section presents the responses to the research questions in each sub-section. Table 2 shows the selected publications along with the title, year, and approach used for testing RESTful APIs.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Ref.** & **Title** & **Testing Approach** & **Year** \\ \hline
[20] & Property-based Testing of JSON based & \multirow{2}{*}{Behavioral testing of RESTful APIs} & \multirow{2}{*}{2014} \\  & Web Services & \\ \hline
[21] & Model-driven Testing of RESTful APIs & \multirow{2}{*}{Abstract model-based testing} & \multirow{2}{*}{2015} \\  & RESTful API Automated Test & & \\  & Case Generation & & \\ \hline
[23] & Study on REST API Test Model & \multirow{2}{*}{Expressive description language-based testing} & \multirow{2}{*}{2017} \\  & Supporting Web Service Integration & & \\ \end{tabular}
\end{table}
Table 2: Selected papers.  
Figure 3: Paper distribution based on quality score.  
RESTful API testing has been regarded as a thriving topic. There have not been many studies published and this means that there is a lot of room for research. The main reason is that it is the flexibility of the methodology being used for developing, deploying, and consuming RESTful APIs. The lack of following one standard for describing the RESTful APIs is one of the hurdles. Hence, many studies are based on a few assumptions to start with.  
To fulfill the objective of answering the inferred research questions, the primary studies were analyzed deeply and relevant data were extracted. The main focus was on the methodology, tools, and approach used for testing RESTful APIs.  
The following were the parameters in line with the data of the study:  
* The top-down approach is followed by each study.
* The methodology is followed for RESTful testing.
* Tools or frameworks are developed for RESTful APIs' testing.
* Any supportive facts are required to answer the research questions.  
Based on the studied facts and analysis, the following sections discuss responses to research questions.  
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Ref.** & **Title** & **Testing Approach** & **Year** \\ \hline  & Automatic Generation of Test Cases for & Automated testing using & 2018 \\
[24] & REST APIs: a & specification-based approach & 2018 \\
[25] & Metamorphic Testing of RESTful & Metamorphic relations-based testing & 2018 \\  & Web APIs & Testing based on parallel API execution & 2018 \\
[26] & Automated API Testing & and sequenced API calls & 2018 \\ \hline  & Producer-consumer dependencies and & & \\
[27] & RESTler: Stateful REST API Fuzzing & dynamic response & 2019 \\  & feedback-based testing & & \\ \hline  & QuickREST: Property-based Test & & \\
[28] & Generation of OpenAPI-Described & Automatic property-based testing based & 2020 \\  & RESTful APIs & on OpenAPI documents & 2020 \\ \hline  & Al-Driven Web API Testing & AI-based testing for RESTful APIs & 2020 \\ \hline  & A simple, lightweight framework for & Communication & \\
[30] & testing RESTful services with TTCN-3 & technology-independent testing using & 2020 \\ \hline  & Differential Regression Testing for & Regression testing targeting contract & \\
[31] & REST APIs & and service changes & 2020 \\ \hline  & RESTTESTGEN: Automated Black-Box & Blackbox testing using a generated unit & \\  & Testing of RESTful APIs & test based on RESTful API’s standard & 2020 \\  & document (OpenAPI) & & \\ \hline  & Deep Learning-Based Prediction of Test & Deep learning-based testing approach & \\
[33] & Input Validity for RESTful APIs & by predicting inter-parameter & \\  & Input Validity for RESTful APIs & dependencies and validity of & \\  & input request & & \\ \hline  & RESTest: Automated Black-Box Testing & Testing using constraint-based and & \\  & of RESTfulWeb APIs & fuzzing oriented generated input & 2021 \\ \hline  & A Black Box Tool for Robustness Testing & Robustness testing based on minimal & \\
[35] & of REST Services & information exposed for RESTful APIs & 2021 \\ \hline \hline \end{tabular}
\end{table}
Table 2: _Cont._