您在這裡

Testing during maintenance

7 九月, 2015 - 12:26

One of the major problems with maintenance is that modifying a system, whether by correcting a component or extending its functionality, there is a high risk of disturbing, unintentionally but unavoidably, functions that do not need to be (and in fact should not be) changed. This accentuates the need for regression testing: testing that not only does the modification work, but everything that was not modified continues to work as before. Any but the most trivial change may require you to rerun an entire system test.

During maintenance, you will create new programs or even subsystems (to address Type 3 SCRs – functional extensions) and you will modify existing programs (typically to address other types of SCRs, but sometimes also Type 3s). Unit testing new programs during maintenance is no different from doing it during development. Testing program modifications, however, can be a little less rigorous. In general, you will test only that the modification works, but perform no regression testing (at the unit level) to check that the unmodified parts still work as specified. This is possible because the risk of errors is relatively low and because the correction-induced errors you may have caused will most likely be caught by the system-test level regression test, which is not optional, whether your maintenance release contains major new functionality or not.

For emergency fixes (type 0), the same rule holds. Regression testing of emergency fixes just isn’t cost-effective. However, in the next scheduled release, the type 0 fix must be backed out and resubmitted as a type 1 fix. This way, it will be re-implemented, unit tested at the program level and regression tested at the release level, just like other SCRs.

Correspondingly, the regression test at the system test level becomes more important. This has several consequences:

  • The regression test is time-consuming and costly. However, it does not cost much more to test many modifications than just a few. This is a powerful argument for organizing maintenance in releases, as described in the previous paragraph.
  • The system test from the first release becomes the regression test for the following releases. This is in fact part of the “product” as delivered to operations; no system is complete without it.
  • When planning a new release,one of the first deliverables is the revised regression test model. For releases that have a lot of new functionality,you may also want to schedule other tests,depending on how you judge the risk of an unsatisfactory result. These tests are likely to be fairly infrequent and also fairly release specific,and the maintenance of a regression test for these purposes is usually not economical.