# API Testing Tools

## Summary


As organizations move towards DevOps, Continuous Integration (CI) and Continuous Delivery/Deployment (CD), there's a need to have quicker feedback from testing. End-to-end or UI testing are slow. API testing is one solution but it must be easy to create, execute and maintain API tests. It needs to be automated as much as possible. To enable this, many tools are available in the market. 

There are both open-source and commercial API testing tools. We can't say which of these is the best. We should select a tool that best suits our project. The selection should be based on features and how well it integrates with other tools and our processes. 

Since 2018, some tools are beginning to adopt AI/ML. This could become an essential feature in the years to come.

## Discussion

### What should I look for when selecting an API testing tool?

At the minimum, any tool should make API requests and validate responses. Other features to consider are: easy to install, learn and customize; good documentation; available on multiple platforms/devices; supports multiple protocols (REST, SOAP, HTTP, JMS, TCP/IP); integrates with CI/CD pipelines; can sniff exploratory tests to automatically create API tests; tests but also validates API specifications. 

The **syntax** for writing tests must be simple enough for those with little coding experience. 

Pick a tool that **integrates** with your automation framework. For example, REST-assured integrates seamlessly with Serenity. 

For teams, the tool should enable **collaboration**. It should be possible to share test cases and results, or publish tests and allow others to run them easily. 

Some tools such as SoapUI are dedicated to API testing. Others such as JMeter was designed for load testing but supports API testing. Swagger is meant for designing APIs but can also be used for testing. This is worth considering if you're looking for single unified tool.


### What are the types of API testing tools?

Broadly, we can identify the following types:

    + **Command-Line**: cURL is an example that can query an API and print out the response. It uses libcurl that can be used in any client-side program. Thus, developers can create their own CLI-based API testing tool using libcurl. Dredd is another CLI tool. On Windows, PowerShell is a good choice.
    + **Browser-Based**: DHC and Postman are examples that can run within a browser. Compared to CLI tools, they offer a user-friendly interface for testers. Postman was initially delivered as a Chrome App, then a standalone desktop app, and then a web app since September 2020.
    + **Standalone IDE**: Katalon Studio is an example that supports test case generation, CI/CD integration, and many types of testing (API, Web, Mobile, Desktop). Karate is an example that can be installed as a plugin/extension and used within IDEs/editors such as IntelliJ or Visual Studio Code. It supports API testing, mocks, performance testing and UI automation.

### Could you describe some API testing tools?

Katalon Studio claims to be "all-in-one test automation solution". It offers free and enterprise versions. Free version supports basic record and playback, REST and SOAP, OAuth for authentication, different types of testing (keyword-driven, data-driven, BDD), parallel execution, and headless execution. It can also import tests from other tools/frameworks such as Selenium, Postman, JUnit, TestNG, SoapUI and Swagger. 

In Postman, tests can be organized into collections that can be shared with others. API design and maintenance can be done within Postman, which serves as the single source of truth. Multiple APIs can be version controlled and tagged. Mock servers and environments can be created. It can monitor API uptime and responsiveness. Newman is Postman's CLI tool for easy integration into CI/CD pipelines. 

An alternative to Postman is Hoppscotch (previously called Postwoman). It runs within the browser and supports GraphQL. As is common, requests can be customized with authentication, headers, and pre-request script. Assertions are in JavaScript. 

OpenAPI.Tools has a list of API testing tools, briefly describing each of them.


### What's the syntax that tools use for specifying API tests?

Karate uses Gherkin syntax that was popularized by Cucumber for BDD. REST-assured also supports BDD syntax. 

Taurus allows tests to be written in YAML or JSON. 

Parasoft SOAtest uses visual drag-and-drop. Drag-and-drop method of creating tests is also supported by SoapUI. In addition, SoapUI supports Groovy language, which has a concise expressive syntax for the Java platform. 

ACCELQ is a cloud-based application for which no code needs to be written. 


### Which are the API testing tools in various languages?

Some tools support many languages. Postman has code generators in many languages that developers can use. RedwoodHQ is another tool that has support for Java/Groovy, Python and C#. Fiddler supports .NET, Java and Ruby. 

API testing in Java domain has been traditionally hard. REST-assured simplifies this with support for XML or JSON format in requests and responses. 

In Ruby, Airborne is an option. It relies on RSpec. It works with APIs written in Ruby on Rails. 

WebInject is a tool written in Perl and therefore requires a Perl interpreter at runtime. 

In JavaScript, Chakram is a tool based on Node.js, Chai.js and Mocha. It uses BDD syntax. Another tool in JavaScript is Frisby that uses Node.js and Jasmine. 

In Python, pyresttest is applicable. It's also a micro-benchmarking tool. Configuration is in JSON or YAML. Hence, little coding knowledge is needed. 


### How is artificial intelligence enabling API testing tools?

Test engineers have always found UI testing to be easier than API testing. They may not know how all the software components work together via their APIs. API testing requires better understanding of the software. AI/ML is attempting to remove some of this complexity. One approach is to simply monitor UI tests, discover patterns and then use this knowledge to automatically create API tests. 

One tool that generates API tests is Parasoft SOAtest Smart API Test Generator. It's installed as an add-on to a web browser. For example, in an e-commerce site, it would learn that a cart ID is important across API calls. A human could indicate that number of items in the shopping cart is important. AI would learn this and validate this in future API calls. 

Other AI/ML testing tools include Applitools, Testim, Sealights, Test.AI, MABL, Retest, and ReportPortal. AI/ML can catch UI changes and identify them as bugs or features; analyze code changes and execute relevant tests; detect code changes and automatically update existing tests; automate result analysis and defect management. 

## Milestones

2000

It's in this decade that APIs start getting popular, partly due to Salesforce, eBay and Amazon. Due to Agile practices, there's also a need to automate API testing. 

2010

In this decade, focus shifts to scalability. Adoption of DevOps and CI/CD make it necessary to embrace automated API testing. However, even by the end of the decade, many organizations continue with UI testing. 

Oct  
2018

Parasoft SOAtest 9.10.6 is released. This includes AI/ML capability that's now part of it's Smart API Test Generator. In general, 2018 is the year when tools start including AI/ML. Focus shifts from just test automation to include test generation.