1. The importance of ‘shift left’ within a development lifecycle
2. Ideas and insights on how to enable testing in a CI Pipeline
3. The benefits of continuous performance testing

Josh Crowther is one of the Lead Developers at Capacitas, where he is responsible for the design and implementation of Capacitas’ flagship products and ensuring that development best practices are adhered to at a high standard across the Development Team. He has worked extensively with Graphical Frameworks, Web Frameworks and APIs. As a Continuous Integration enthusiast, Josh has defined and created the build and release pipelines that the Development Team use daily, to enable frequent product releases for the company’s customers. More recently, he has transitioned tightly coupled source code for one of Capacitas’ products, into a tiered application which is indifferent to its presentation layer. This has enabled Capacitas to push forward into the DevOps domain and open many doors for the company.

As DevOps becomes increasingly popular when delivering a software solution, it has become even more important to provide Performance Testing with the breadth and depth of analysis required in demanding timeframes.

We have faced a range of challenges whilst trying to address the high demand for stable code with small time deltas; the ever-changing agile world means that we constantly need to be one step ahead and ensure that we “shift-left” as early as possible, which is difficult when the process is not autonomous. It became clear to us, that we must automate parts of our Performance Testing methodology to produce reliable and fast test results.

The concept we developed upon was simply an analysis engine, which was provided test results to produce rapid analysis on the datasets and alert the user of any issues in the Performance Test that was performed. This solution was satisfactory, but as time went on, the demand for less manual intervention rose. Now as it stands, our solution has zero manual intervention and has been rebuilt to integrate with Continuous Integration pipelines. Thus, our solution has allowed us to have a better feedback loop with our clients as well as internally as a company and has improved our Performance Testing framework. Additionally, it has aided us in ensuring we deliver crucial performance tests early on in sprint cycles rather at the end (“Shifting Left”).

Our presentation looks at our approach to successfully instrument a Performance Testing methodology for Continuous Integration; Including, the ideas and inspiration we had when designing our analysis solution and our mechanisms to determine whether code is stable. We intend to dive into the issues we faced from our clients, likewise internally and we will demonstrate our framework to enable Performance Testing in Continuous Integration.

 

---

Callum Nicolson is a consultant at Capacitas, involved in implementing Capacitas’ Performance Engineering processes within a number of large organisations. This process ensures performance testing is embedded within the lifecycle of the project from its inception, allowing for the early detection of performance defects before integration where they become harder to identify and resolve. Callum has harnessed automated analysis tools to analyse vastly larger data sets than possible manually, while reducing analysis time providing faster, more reliable results allowing the client to make better informed decisions. Callum is an advocate of the “shift left” methodology using in-house tools to integrate automated alerting into client’s Continuous Integration pipelines, allowing for immediate detection of performance defects as new code is developed, reducing testing overhead and escalation costs.