The quality is being tracked on the convergence of meeting the release criteria such as a number of tests executed/passed/open P1/P2/P3 bugs, Stress/Longevity criteria, Performance targets, code coverage, etc.
To track the quality, below are some of the measurements done on each build or sprint or weekly basis.
- Test Execution metrics: track the dev code changes and testing progress since the last build or tracking period.
- Bug Metrics/Defect Tracking: the number of open bugs and closed bugs since last build or tracking period.
- Code Coverage: %class, %method, %line coverage since the last build or tracking period.
- Performance Numbers: throughput numbers since the last build or tracking period.
- Bugs metrics containing incoming and fixing of bugs would be plotted as graphs and tracked on a weekly basis. A bell curve is what expected for the period of product testing cycle. That means initially, incoming or new bugs would be low and later in high and then later goes down again as fixing bugs. If deviated from this bell curve, then there might be the possibility of low quality as bugs keep coming, and convergence of release is not meeting the criteria.
- The quality is assessed and projected at different levels than in absolute numbers. These are –
- High quality
- Medium quality
- Low quality
The high-quality product should be the target of any good product and of course, it would cost more compared to a low-quality product.