Image courtesy of NordicAPIs.com
API testing is a growing necessity. An IDC study indicated that, at the beginning of 2016, the API Market was tracking a 56% growth rate on a yearly basis. The size of that market totaled over $333 million. With the surge of publicly available API’s, developers will find their API’s being utilized on an exponentially wider scale. Clearly, ensuring that API’s you’ve developed are functioning and bug-free out of the box is now more important than ever. Lets take a look at some API Testing best practices to ensure your services are working as intended.
Documentation
As with any type of testing, how your business requirements and test cases are defined will determine the success of your tests. This becomes especially pertinent when there is no front end GUI to reference. For example, a form that you are filling out with data to be sent may contain a dropdown with specific options, that would not be visible in an API request body. Referencing a document that clearly specifies each field (and associated parameters) being sent through allows a tester to cover all necessary use cases, and minimizes the risk of missing an essential test flow. A popular tool for creating and managing API documentation is Swagger – you can define documentation that makes sense for both developers and other stakeholders (project manager, business analyst, etc.).
A sample Swagger documentation for a file using Swagger-UI.
You can also think of Swagger as a blueprint/planning tool for your API – you can approach from the ground up, and use Swagger to document what you currently have after you’ve already developed your API, Alternatively, you can take a “design first” approach, which would entail mapping out the API before actually doing any development- and then coding to fill in the blueprint.
Automation
The benefits of automation are no secret at this point; reusable, accurate, scaleable tests that provide exponential time/cost savings & increase in coverage – you’ve heard it before. Automating testing for APIs are no exception. Having a solidified automation framework in place will allow you to quickly scale up and modify/add tests as requirements grow or change. Additionally, executing API automation is much more streamlined and quicker than GUI automation – without a graphical interface to poll, tests will simply have an easier time running. Variables like the browser, page objects, frameworks, etc. that are prone to breaking are no longer in the picture, meaning tests will require much less continued maintenance.
A white paper from Paul Grossman compared labor cost of manual testing vs automation – using the cost and hours of testing with 10 manual resources, vs 3 automation engineers. The results showed a dramatic reduction of cost/hour utilizing the 3 automation resources. The automation coverage also detected bugs earlier in the testing cycle – which is a huge timesaver. Per the study:
“Coding defects found post-release cost 5x more to fix than those found during unit testing.”
Again, the goal isn’t necessarily to have 100% automation – manual testing still offers a slew of benefits (see our post on 8 reasons why manual testing is still relevant). Some scenarios are just not feasible to automate, due to complexity/technological limitation, or resource cost of creation greatly outweighing the cost a simple manual test.
Prioritization
As with front end tests, there are a multitude of test types for APIs – your first priority should also be your core functionality. Time spent developing flows for negative tests and edge cases can easily pile up, and (though still important) should be addressed after ensuring your basic request/responses are working as intended. Function calls should also be categorized in a way that will make it efficient for testers both new and seasoned. Moving onto load testing after will allow you to gauge how well your (now correctly functioning) API handles demand in larger volumes.
Structure
Beyond simply ensuring that your functional level services are working, you will need plan and test for specific scenarios. These will go beyond simply running a GET request for a list. An initial approach you can take is syntax testing – which will check if the correct inputs are accepted, and the wrong/incorrect inputs are properly rejected. Filling in fields with negative data types – fields that are optional would be expected to pass positively.
After, you can step into your Functional testing – this will check that your actual method/operation is working as expected. After you know that both your functional/syntax tests work as intended, you can test full flows in the form of scenarios – such as a Create Customer flow for CRM, which would end up utilizing your GET, POST, and DELETE requests.
Looking Forward
Currently, there is still a range of quality and reliability across APIs. Some are well documented and fully functioning, some providing just enough knowledge for only the developers to make sense of. The continuing growth of publicly available APIs means continual improvement to existing applications – similar to the effect that the growth of open source software is continuing to have. As always, more APIs = more to test, so ensuring you’re holistically covering both your automation & manual testing is essential as ever.
