An Initiative to Improve Quality of Software Testing Techniques and Calculating Total Number of Failures using Bayesian Method

Vaibhav Sancheti, G. Srikari Sharma

The current concern regarding quality of evaluation performed in existing studies reveals the need for methods and tools to assist in the definition and execution of empirical studies and experiments. This paper discusses those issues specific for evaluation of software testing techniques and proposes an initiative for a collaborative effort to encourage reproducibility of experiments evaluating software testing techniques . This paper also proposes the development of a tool that enables automatic execution and analysis of experiments producing reproducible research compendia as output that is, in turn, shared among researchers. There are many expected benefits from this endeavour, such as providing a foundation for evaluation of existing and upcoming STT, and allowing researchers to devise and publish better experiments. Total number of failures of a software system can help practitioners to have a better understanding of the software quality. This paper proposes a model to predict the total number of software failures in a software system by analyzing the failure data from testing using failure data and code coverage are combined in a Bayesian way. The methodology is applied to real world failure data to validate its predictability. The predictive accuracy of our model is also evaluated with different methods. The results of our experiment show that our proposed model can provide a very good estimation of the total number of failures. The estimation is stable from a very early point on.