We test a software program, an application, an entire business information system to detect errors—things that don’t work the way they are supposed to. If the system doesn’t work properly, we find out why and fix it (or abandon it, if we judge it too expensive to fix).
We test other things, too. New airplanes are tested before they are declared airworthy. Before buying a new car, you probably want to test-drive it. In the United States, the Food and Drug Administration mandates extensive tests, lasting several years, of any new medication before it can be prescribed or sold over the counter. Sometimes, the tests are insufficient, as with the case of the X-ray equipment that would, under certain circumstances, irradiate patients with 1,000 times the normal dose; as a result, patients died. In other cases, it isn’t possible to test a product under realistic conditions: the Lunar Excursion Module could not be tested on the moon before the real moonshot.
Software—beyond a few hundred instructions—becomes so complex that its behavior cannot be accurately predicted just by inspecting it. Errors creep in through inattention, misunderstanding, or other human weaknesses. Just as no one would want to fly an untested plane, no one would want to use untested software.
Testing is a part—a critical one—of a more general process called software defect removal. 1 (Other techniques include inspection, prototyping, and automated syntax checking.) Even used in conjunction, these defect removal techniques cannot guarantee defect-free software. The most thorough of tests cannot address all the possible values of input data against all the possible states of the database. A defect may remain latent for several years until some highly unlikely combination of circumstances reveals it. Worse, the test itself may be defective: after all, the test, too, is designed and executed by fallible human beings.
In traditional systems development methodology, testing is seen as a discrete activity, which occurs once development is substantially complete. This view causes problems, because the cost of correcting errors becomes exponentially larger the more time passes during which the error remains undetected. An error that is discovered by its author as it is committed can be corrected immediately, at no cost. A requirements error discovered during operation may invalidate the entire system – and carry business interruption costs in addition. Experts estimate that correction costs can be a hundred times higher when the error is discovered after the system has been delivered. The reason for these high costs are that the longer the time between committing an error and discovering it, the more work will have been done and therefore have to be corrected or scrapped and recreated.
- 1316 reads
