What To Look For in a Test Automation Tool

When my team had to pick an API testing tool, we went with Postman. And the more I used it, the more frustrated I became with it. Both maintaining the tests and figuring out why tests failed (especially when in a CI pipeline) took too much effort. So I decided to write my own tool. Or rather, I set to work with pytest and the requests library, and ended up with a framework of just 300 lines of python code that fit our needs.

The core idea behind my framework is that I don’t want a testing tool that only focuses on making it easy to build tests. It should also make it easy to analyse the test results. It should also make it easy to read and understand the tests, to share them, and to maintain them. Because the thing is, during most of those activities I don’t want to be aware of the tool. I want to focus on the tests and what their results tell me about the quality of the product. That’s the key idea here: only at very specific times do I want to focus on the tool as a tool. At all other times it should just help me test better.

In my talk I will use my framework to illustrate these ideas and expand on them. I will also show how these ideas apply more generally than only to API testing frameworks. Since you’re only as good as your tool and a tool is only as good as its user, being able to evaluate testing tools is an important skill for any tester.