An Empirical Study on the Persistence of SpotBugs Issues in Open-Source Software Evolution

Background. Static analyzers can be useful to software developers in detecting and locating code issues and, in addition, classifying their nature. The main problem of static analyzers, however, is that they may signal too many false alarms. Objective. In this paper, we investigate whether code issues that are detected by SpotBugs persist in software code, or if they get removed. We chose SpotBugs because it is one of the best-known and most used static analyzers. Method. We carried out an empirical study on five open-source Java programs and took into account two versions of each of them, to check whether the issues signaled by SpotBugs on the older version had been removed by the time the newer version was released. A total of 1,006 issues were signaled by SpotBugs. Results. Our results show that about half of the issues signaled disappeared between the two versions, but the correction rate was uneven across projects. Issues about the correctness of software code were more likely to be no longer present in the newer version than other types of warnings. Conclusions. Further investigations are required, to understand why some projects appear more active than others in correcting SpotBugs issues, and why very few high-severity warnings were observed in the analyzed code. Nonetheless, the fact that about half of the issues flagged by SpotBugs were removed indicates that the tool is effective in detecting incorrect or otherwise problematic code.