OSS-TMM: Guidelines for improving the testing process of open source software

Open Source Software (OSS) products do not usually follow traditional software engineering development paradigms. Specifically, testing activities in OSS development may be quite different from those carried out in Closed Source Software (CSS) development. As testing and verification require a good deal of resources in OSS, it is necessary to have ways to assess and improve OSS testing processes. This paper provides a set of testing guidelines and issues that OSS developers can use to decide which testing techniques make most sense for their OSS products. This paper 1) provides a checklist that helps OSS developers identify the most useful testing techniques according to the main characteristics of their products, and 2) outlines a proposal for a method that helps assess the maturity of OSS testing processes. The method is a proposal of a Maturity Model for testing processes (called OSS-TMM). To show its usefulness, the authors apply the method to seven real-life projects. Specifically, the authors apply the method to BusyBox, Apache Httpd, and Eclipse Test and Performance Tools Platform to show how the checklist supports and guides the testing process of these OSS products.