CAST 2019

Our agile teams struggled to continuously merge local code changes to the master repository. In our company, our automated test were taking 30 minutes of execution time and the occurrence of flaky tests was just multiplying this time and reducing the confidence in the results. This slowed our agile teams down tremendously to the point where they couldn’t deliver continuously with automated pipelines. Ensuring high quality for each iteration became a challenge for our agile teams as they had to wait for my analysis of the failing tests. We had created automated checks to make our testing more efficient, but instead they were slowing us down and causing bottlenecks - I was a bottleneck.

Our automated tests were maintained by various team members and some of them were not following leading practices - which led to flaky results due to such things as - concurrency relying on non-deterministic behaviour, caching, dynamic content and many other factors. I needed to find out the root causes. I started my investigation and found the common issues were because of environment, locators, coding practices and a lack of knowledge sharing and code reviews. I improved our locators, coding practices, debugging and simultaneously the developers helped us to fix all environment issue and slow page load times, it was a long journey but it was definitely worth it.

Currently our automated test suites takes just 5 minutes to run in our CI environment - a huge improvement! Now my team members are more encouraged to see why a test is failing because the majority of failures now are genuine, and are providing valuable information. As a team, we get to spend more time testing and creating new automated tests, which has resulted in finding more bugs. We’ve also put things in places to avoid ending up in a flaky nightmare again, such as a Wiki Page of leading practices.I’m no longer a bottleneck, the team works together to maintain fast valuable automated tests, and I want to share the insights from that journey with you.

After this session, you’ll learn techniques to:
1. Treat test code as production code and test environments as production environments
2. Improve the stability of your tests which in turn improves your continuous delivery pipeline
3.  Make tests faster to enable fast feedback