# RESTful API Testing Methodologies: Rationale, Challenges, and Solution Directions
## 4 Results
### RQ1--What Are the Main Challenges in Generating Unit Tests for RESTful APIs?  
Going through all primary studies and analyses yielded the fact that there are several challenges in generating unit tests for RESTful APIs. In both [25] and [27], the authors argue that effective and concise testing is the most challenging, whereas in [25], the authors argue that the lack of a test oracle is the biggest hamper to generating an effective unit test, as they are not always available. As for [26], the authors argue that the time taken is a major issue, and the authors of [33] argue that learning whether the input is valid or not before executing the REST call is crucial. The authors of [34] report that the lack of source code is the biggest issue. Looking at the challenges in a more consolidated approach for the sake of categorization, the following issues emerged as the most common ones:  
* Security plays a vital role in keeping the APIs safe and stable. However, at the same time, this poses a big challenge in generating unit tests and testing in general. The usual approach of unit test generation depends on an API description that is usually done using OpenAPI, or by using any other compatible descriptive language. This methodology is used by black-box testing because access to the source code is not available. The descriptive document does not contain any security information. Because of such a scenario, the unit test generation process is either limited to fewer unit tests or ends up with invalid unit tests because the required security information was not incorporated into the generation process.
* Non-compliance to description document standards is another challenge in unit test generation. The most important catalyst for auto-generating unit tests for RESTful APIs is the description document. Different organizations are maintaining this document in different ways. XML, plain JSON, and OpenAPI are some of the document standards. RESTful APIs are quite flexible in this regard, which is another cause of the emergence of this challenge. Because there is no compulsion for maintaining a standard descriptive document for RESTful APIs to work, not much importance is given to this aspect. As a result, the standard tools and frameworks are sometimes unable to even proceed or end up with faulted unit tests.
* An inconsistent descriptive document contributes to the list of challenges as well. Studies showed that, many times, organizations are not keeping the descriptive document for RESTful APIs up to date. As a result, the APIs that are in production are different than the descriptive document. Rapid changes and pressure to make modified APIs available as early as possible in production pave the way to the descriptive document being inconsistent. Hence, the unit test generation produces incomplete or inconsistent tests.
* Complex input types are yet another challenge with which to deal. Modern-day RESTful APIs enable communication between hybrid networks, backends, and all possible kinds of devices. Data sent over could be of possibly any time, and ranging from simple integers and strings to a whole file or group of files. One of the core parts of any unit test generation framework is to infer test input values, which is very hard when complex types are involved. For this reason, several studies, such as [32], exclude the scope of complex types when generating unit tests.