The Unexpected Truth About UI Test Automation Pilot Projects

In our contact with organizations focused on delivering higher-quality solutions to customers, often we come across teams that struggle to build up their test automation proof of concept or, worse, fail to get on the right foot with UI test automation at all. Implementing test automation is a complex, intricate process that eats up a lot of time and resources, making it difficult to analyze the factors that lead to the success or failure of a project. We wanted to gain a better understanding of what it takes to be successful in the UI test automation field, so we can better guide our customers on a path to success with their automation projects. To do this, we conducted a survey of 985 IT professionals and asked them about their experiences in developing test automation projects.

Who Participated?

The 985 IT professionals that make up the survey respondents work in various job positions and with varied technical backgrounds: Based upon their background and experience, we divided the respondents into two subgroups: testers and developers. We then analyzed their responses to help determine what makes test automation projects succeed or fail.

What We Learned

You can view the complete analysis of the survey results in our convenient leaf-through report. Our goal was to explore the first steps teams in the field of automated functional testing take, as well as where they are today with their automation efforts and what helped them get there. We discovered a recurring theme: software developers are a vital component of successful UI test automation projects. Involving software developers early on when preparing applications for automation helps most teams be successful. In fact, the majority are successful not only with pilot projects but with UI test automation in the long run. Key findings:

• Most successful test automation pilot projects involve developers.
• Pilot projects determine the future of automated testing within organizations. Most companies that fail will not try again.
• Professionals with coding experience were most successful at turning failed projects around.

Specifically, we found three primary factors that contributed to the success of automated testing projects:

1. The development team helped make the application UI automatable;
2. Management had realistic expectations;
3. Communication between the automation lead and management was effective.

Since a failed pilot project can doom the adoption of automated testing within your organization, it’s especially important that you set up your project to succeed the first time.

Takeaways

How do you draw from these success factors when implementing an automated testing project at your company? In the report, industry expert Jim Holmes, VP of ALM and Testing at Falafel Software, offers some guidance based upon the survey results. Specifically he tackles how to ensure that your team has the right culture to succeed, how to set the right expectations for the project and how to get developer buy in. Get your copy of the free 2014 Report on UI Test Automation Pilot Projects. Enjoy the read!