When to automate software testing? A decision-support approach based on process simulation

Software test processes are complex and costly. To reduce testing effort without compromising effectiveness and product quality, automation of test activities has been adopted as a popular approach in software industry. However, because test automation usually requires substantial upfront investments, automation is not always more cost-effective than manual testing. To support decision-makers in finding the optimal degree of test automation in a given project, we recently proposed a process simulation model using the System Dynamics modeling technique and used the simulation model in the context of a case study with a software company in Calgary, Canada. With the help of the simulation model, we were able to evaluate the performance of test processes with varying degrees of automation of test activities and help testers choose the most optimal cases. The goal of the earlier study was to investigate how the simulation model can help decision-makers decide whether and to what degree the company should automate their test processes. In this article, we present further details of the System Dynamics model, its usage scenarios and examples of simulation experiments independent from a specific company context. Copyright (c) 2015 John Wiley & Sons, Ltd.