Automated Combinatorial Testing for Software (ACTS)
Researchers at the National Institute of Standards and Technology (NIST) and the University of Texas, Arlington have released a variety of briefings, reports, and results concerning the automated combinatorial testing for software. Many of the major news feeds were carrying the stories yesterday.
After reviewing their reports and findings, it sounds like good news for automated testing
systems.
There never has been enough time or money to do all the testing that could or should be done for any non-trivial project. But these results at least provide some support to the idea that a substantial majority of software defects can be effectively captured by automated testing with something less than full-exhaustive testing. See the following link for the full article, the reports and the results: http://csrc.nist.gov/groups/SNS/acts/index.html
It is normal in the software industry to adopt approaches to testing that cover the most commonly used features and cases. In my experience, you normally get the most bang for the buck by focusing on expected use cases.
Still, these results were encouraging enough to motivate me to look into the new tool to be released to see how I like it. I sent Rick Kuhn a note today to get some info. I will post the results of my assessment of the tool when I am finished.
-Stu