Uma investigação do custo envolvido na identificação manual de mutantes equivalentes

Abstract

Mutation testing consists of changing the software system to be tested by applying mutation operators. As a result, applying such operators generates different versions of this system. These versions are called mutants. Mutants are used to verify that test cases built for the software system are able to identify these changes made to the original code by checking the quality of the test suite. If unit tests are able to detect all changes, the mutants are killed and the test set is sufficient; If some mutants are not killed, analysis of these mutants is required to

check whether new test cases are needed to detect changes or whether live mutants are equiva- lent to the original code, ie generate the same output as the original code. The problem with

mutation testing is the cost involved in identifying whether these mutants are equivalent to the

original software system. Identification of equivalent mutants is undecidable, so analysis is usu- ally conducted manually, which consumes hours of an analyst. Thus, this research investigated

the human cost involved in the manual analysis of mutants to verify equivalence between the original and mutant codes. To verify the manual cost of analysis, the DiffMutAnalyze tool was developed in order to compare two types of analysis: using DiffMutAnalyze with the manual, ie

without the use of a specific tool to aid inspection of mutants with the original codes. DiffMu- tAnalyze includes the Major tool for mutant generation, so only Java projects can be inserted

into the tool, as Major is a tool for mutation in Java projects. Through the experiment, it is possible to verify the time spent with the manual analysis and to verify the difference between this analysis and the analysis by means of a computational support tool. The purpose of this tool is to enable original and mutant programs to be compared side by side, facilitating the manual analysis of the mutants, since in the tool the location of the mutation is evident to the user. To perform this comparison of the analyzes manually and through DiffMutAnalyze, an experiment was conducted. As a result of the experiment, in general, the participants took an average of 31 minutes and 30 seconds to perform manual mutant analysis, when the analysis was performed using DiffMutAnalyze, the average time was 14 minutes. Thus, the analysis performed using DiffMutAnalyze was reduced by more than 50% of the time. Therefore, the use of a tool to assist in the identification of equivalent mutant identification reduces the cost of this analysis. In addition, it has been found that the tool assists in performing the mutation test as the original code mutation is performed automatically and the surviving unit test mutants are made available for inspection. This makes the comparison of the original and mutant codes faster because in the tool the codes are automatically made available and the change made is identified by DiffMutAnalyze. Thus, the analyst need not look for the location of the change made by the mutation operators.