In order to do Agile ‘properly’ we need test automation – even more so with DevOps. API test automation can be reliable and is often the first place to start. User Interface test automation however, is known to be fragile and this fragility or flakiness is discussed a great deal online and in conferences presentation.
Why is it fragile? What is this flakiness due to? We look at the life of an automated mouse.click(). We look at what could go wrong and then draw conclusions about test automation and the way we should build it.
We will draw on these conclusion to discuss the Test Automation Paradox which is that when we are testing a system with automated tests we are testing two things: (i) the system under test and (ii) the automated tests themselves.
We will see how the size of the system under test’s input space is critical for the automation approach.
We will draw conclusions about test code reliability. What level of reliability do we need for our .click()? How do we get high reliability and low fragility? What could possibly go wrong?