Measure Conditional Diversity

Conditional Diversity is a measure of control dispersion/variation at the source-code level. The true/false evaluation frequencies of conditional expressions in conditional statements are used to measure the control dispersion of a test suite. These evaluation frequencies could be un-evenly distributed for a particular test suite. For example, the true branches in the code could be more heavily exercised than the false ones. Therefore, conditional diversity points to portions of source code with high and low test concentration. Conditional diversity is used to determine if you are gaining false confidence in your testing by running the "same" or "similar" test over and over again. Use conditional diversity to determine where in the code you need to apply balancing and skewing schemes to diversify your testing and increase chances of defect detection.

Conditional diversity is expressed as a conditional diversity vector, where each value in the vector is the conditional diversity of a particular conditional expression in the code. Executions of the program on multiple test cases give a conditional diversity matrix, which consists of conditional diversity vectors, each vector corresponding to a particular test case. The conditional diversity matrix is used to calculate the standard deviation vector, which is interpreted as a measure of dispersion of conditional diversities relative to the conditional diversity mean value. More dispersed the conditional diversities are among test cases, higher control program-execution variation results from the test cases.

To learn more about conditional diversity, including a concrete example, go to diversity.

Measure Data Diversity

Data Diversity is a measure of data dispersion/variation at the source-code level. Data Diversity is used to determine if you are testing your code with the same, similar or different internal data. Testing with different data at the GUI/interface level might result in arbitrarily same or similar internal-program data, which would give you false confidence that your testing is data diversified. It is highly desirable to know the internal data distribution involved in testing, to take steps to increase it, and to continually measure it, improve it, and diversify it. Higher internal program data diversity indicates high data variation among test cases, whereas a lower data diversity indicates the code is covered with the same or similar data over and over again.

Control and data are tightly related with respect to test diversity. For example, branch selection in the code is governed by values of program variables, and vice versa, the values of program variables are governed by branch selection. If two test suites result in different conditional diversities, then the internal data states involved in the test suites are different, in turn, resulting in different data flowing throughout the program. In effect, conditional diversity is used to measure data diversity.

Data diversity is calculated as an average of the individual data diversities for each conditional statement. The individual data diversities are calculated as a percentage of test suites for which a conditional expression has distinct conditional diversities. If two test cases have different conditional diversities then they execute different paths in the code, which is only possible if different data flows throughout the program. Therefore, higher conditional diversity means that the internal-program data involved in testing is more diverse.

The conditional diversity matrix is used to calculate the data diversity vector, where each value in the vector is the data diversity of a particular conditional expression in the code. The conditional diversity matrix contains the conditional diversity vectors associated with each test case. The data diversity vector is calculated by comparing the corresponding values in the conditional diversity vectors to determine the percentage of distinct conditional diversity values.

Higher data diversity could be obtained by adding more test cases, by reordering existing test cases, and by replacing existing test cases with new ones utilizing balancing and skewing.

To learn more about the theory and the practice behind conditional diversity, including a concrete example, go to diversity.