Measure Conditional Diversity
is a measure of control dispersion/variation at the source-code level.
The true/false evaluation frequencies of conditional
conditional statements are used to measure the control dispersion
of a test suite. These evaluation frequencies could be un-evenly distributed
for a particular test suite. For example, the true branches
in the code could be more heavily exercised than the false
ones. Therefore, conditional diversity points to portions of source
code with high and low test concentration. Conditional
diversity is used to determine if you are gaining false confidence
in your testing by running the "same" or "similar"
test over and over again. Use conditional diversity to determine where
in the code you need to apply balancing and skewing schemes to diversify
your testing and increase chances of defect detection.
diversity is expressed as a conditional
diversity vector, where each value in the vector
is the conditional diversity of a particular conditional expression
in the code. Executions of the program on multiple test cases
give a conditional
diversity matrix, which consists of conditional diversity
vectors, each vector corresponding to a particular test case.
The conditional diversity matrix is used to calculate the standard
deviation vector, which is interpreted as a measure
of dispersion of conditional diversities relative to the conditional
value. More dispersed the conditional diversities are among
test cases, higher control program-execution variation results
from the test cases.
To learn more about
conditional diversity, including a concrete example, go to diversity.
Data Diversity is a
measure of data dispersion/variation at the source-code level. Data
Diversity is used to determine if you are testing your code with the
same, similar or different internal data. Testing with different data
at the GUI/interface level might result in arbitrarily same or similar
internal-program data, which would give you false confidence that your
testing is data diversified. It is highly desirable to know the internal
data distribution involved in testing, to take steps to increase it,
and to continually measure it, improve it, and diversify it. Higher
internal program data diversity indicates high data variation among
test cases, whereas a lower data diversity indicates the code is covered
with the same or similar data over and over again.
Control and data are
tightly related with respect to test diversity. For example, branch
selection in the code is governed by values of program variables, and
vice versa, the values of program variables are governed by branch selection.
If two test suites result in different conditional diversities, then
the internal data states involved in the test suites are different,
in turn, resulting in different data flowing throughout the program.
In effect, conditional diversity is used to measure data diversity.
Data diversity is calculated
as an average of the individual data diversities for each conditional
statement. The individual data diversities are calculated as a percentage
of test suites for which a conditional expression has distinct conditional
diversities. If two test cases have different conditional diversities
then they execute different paths in the code, which is only possible
if different data flows throughout the program. Therefore, higher conditional
diversity means that the internal-program data involved in testing is
diversity matrix is used to calculate the data
diversity vector, where each
value in the vector is the data diversity of a particular conditional
expression in the code. The conditional diversity matrix contains
the conditional diversity vectors associated with each test
case. The data diversity vector is calculated by comparing the
corresponding values in the conditional diversity vectors to
determine the percentage of distinct conditional diversity values.
Higher data diversity
could be obtained by adding more test cases, by reordering existing
test cases, and by replacing existing test cases with new ones utilizing
balancing and skewing.
To learn more about
the theory and the practice behind conditional diversity, including
a concrete example, go to diversity.